From this KB it is clear that we can pivot an element about any arbitrary pivot point in space. 

This is Ok for intermediary optical elements in the optical chain but breaks when you want to pivot either the Object of Image surface.

Here is a an example showing a simple Optic Fiber with Chief ray centered on to Image surface. How can I pivot Surfaces 1→3 about the Pivot point centered on the Image surface?