Question

How to speed up the optimization of Zemax NSC through Python programs

Cheng-Mu Tsai

Hello,
I have a question about NSC by Python program optimization.
The program must iteratively run the local optimization defined by Zemax.
However, after running some local optimizations, the optimization becomes very slow.
How to speed up the optimization of Zemax NSC through Python programs?

12 replies

David.Nguyen
Rank
Userlevel 7
Badge +2

@Cheng-Mu Tsai 

 

Do you have an example of your code to share so we can reproduce the issue? Its hard to say how a program can be optimized without seeing it (at least for me). From your description of the problem, are you opening and closing the optimization tool at every iteration of the loop? It could be then that some resources (cores) become locked somehow and are not available for the following iterations. I’m just wondering at this point, but if that’s what you are doing, could you instead open the optimization tool once before the start of the loop and only close it after the loop is done. Obviously, if you need to open another tool during an iteration, this will not work (only one tool can be open at a time).

Take care,


David

Cheng-Mu Tsai

Hi, David, 
Thanks for your reply. 
When we optimize the coefficient of the freeform, 
we always start from the low order (e.g., radius) to high order term (e.g., R^4, R^6,R^8, ...).

step 1 From the begining, the low order terms are set to be variables and do the optimization. 
step 2 Then the high order terms would be set to be variables and do the optimization Sequentially. 
step 3 Then optimize other lens, go back to step 1. 

Therefore, the optimization must be open and close every time. 
The following is my optimiztion function. 

def optimization(TheTOOL):
    LocalOpt = TheTOOL.OpenLocalOptimization()
    #Automatic=0, Fixed_1_Cycle=1, Fixed_5_Cycles=2, 
    #Fixed_10_Cycles=3, Fixed_50_Cycles=4, Infinite=5  
    LocalOpt.Cycles = ZOSAPI.Tools.Optimization.OptimizationCycles.Fixed_5_Cycles
    LocalOpt.RunAndWaitForCompletion()
    LocalOpt.Close()  

I know that the running speed depends on the number of the variables. 
But when the program run other lens, the running speed is slow down very much even shut down. 
Could any method speed up the running speed? 

 

Best wishes, 

CM tsai

David.Nguyen
Rank
Userlevel 7
Badge +2

@Cheng-Mu Tsai 

 

You could also do the tool opening and closing outside this optimization function. In pseudo code you could do:

def optimization(theOptimizer):
    ...
    theOptimizer.RunAndWaitForCompletion()

def main():
    theOptimizer = TheSystem...

    for:
        optimization(optimization tool)

    theOptimizer.Close()

I got confused a bit, are you saying that the optimization is slow in general or it gets slower as the code runs?

Take care,

 

David

jwbeenakker
Userlevel 3

In addition to @David.Nguyen ‘s comment on sharing the code (for me a bit more of context code would certainly help), do you see any difference in utilisation of you computer hardware (eg. more and more memory being allocated, CPU usage, etc)?
 

Cheng-Mu Tsai

Hi, jwbeenakker and David.Nguyen, 
Thanks for your reply. 
The following is the pseduo code. 


def setMFETarget(imgName,TFlux,TheMFE,opBng,ncol,nrow):
    #給定影像，設定MFE偵測器上target值
    TheMFE.GetOperandAt(opBng-1).Target=TFlux 
    img = cv2.imread(imgName)  # 讀取圖片
    imgGray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY);
    pxFlux=float(TFlux)*imgGray/imgGray.sum()
    for i in range(nrow):
        nn=i*ncol
        for j in range(ncol):
           TheMFE.GetOperandAt(nn+j+opBng).Target=pxFlux[i][j] 
    return pxFlux

def getRandomPar():
    scl=0.475
    Rx=random.uniform(90, 130)
    Ry=random.uniform(35, 70)
    #Kx=random.uniform(-1, 1)*10**(random.uniform(0, 4))
    #Ky=random.uniform(-1, 1)*10**(random.uniform(0, 4))
    rndPar=[0,Rx*scl,Ry*scl,Rx,Ry,0,0]

    return rndPar 

def optimization(TheTOOL):
    LocalOpt = TheTOOL.OpenLocalOptimization()
    #Automatic=0, Fixed_1_Cycle=1, Fixed_5_Cycles=2, 
    #Fixed_10_Cycles=3, Fixed_50_Cycles=4, Infinite=5  
    LocalOpt.Cycles = ZOSAPI.Tools.Optimization.OptimizationCycles.Fixed_5_Cycles
    LocalOpt.RunAndWaitForCompletion()
    LocalOpt.Close()  

def setParTo0(m1):
    xidBng=14
    yidBng=30
    zidBng=46
    nOder=16
    for i in range(nOder):
        m1.GetObjectCell(parN[i+xidBng]).DoubleValue = 0
        m1.GetObjectCell(parN[i+yidBng]).DoubleValue = 0
        m1.GetObjectCell(parN[i+zidBng]).DoubleValue = 0
           
def optmSingle(TheSystem,parN,opBng,ncol,nrow,cnt):
    TheNCE = TheSystem.NCE
    #TheMFE = TheSystem.MFE
    TheTOOL = TheSystem.Tools
    
    xidBng=14
    yidBng=30
    zidBng=46
    nOder=16
    
    rndPar=getRandomPar()
    m1=TheNCE.GetObjectAt(1)
    for i in range(1,7):
        m1.GetObjectCell(parN[i]).DoubleValue = rndPar[i]    
    setParTo0(m1)
    optimization(TheTOOL)

    for i in range(nOder):
        print('optimization =',i,datetime.now())
        Solver=m1.GetObjectCell(parN[i+xidBng]).CreateSolveType(ZOSAPI.Editors.SolveType.Variable)
        m1.GetObjectCell(parN[i+xidBng]).SetSolveData(Solver)
        Solver=m1.GetObjectCell(parN[i+yidBng]).CreateSolveType(ZOSAPI.Editors.SolveType.Variable)
        m1.GetObjectCell(parN[i+yidBng]).SetSolveData(Solver)
        Solver=m1.GetObjectCell(parN[i+zidBng]).CreateSolveType(ZOSAPI.Editors.SolveType.Variable)
        m1.GetObjectCell(parN[i+zidBng]).SetSolveData(Solver)
        optimization(TheTOOL)

    print('Variable Fixed =',i,datetime.now())
    for i in range(nOder):
        Solver=m1.GetObjectCell(parN[i+xidBng]).CreateSolveType(ZOSAPI.Editors.SolveType.Fixed)
        m1.GetObjectCell(parN[i+xidBng]).SetSolveData(Solver)
        Solver=m1.GetObjectCell(parN[i+yidBng]).CreateSolveType(ZOSAPI.Editors.SolveType.Fixed)
        m1.GetObjectCell(parN[i+yidBng]).SetSolveData(Solver)
        Solver=m1.GetObjectCell(parN[i+zidBng]).CreateSolveType(ZOSAPI.Editors.SolveType.Fixed)
        m1.GetObjectCell(parN[i+zidBng]).SetSolveData(Solver)
    print('Over 1 OPMT =',i,datetime.now())
    

if __name__ == '__main__':    
    for i in range(3):
        ss=str(i).zfill(4)
        imgName=f'p{ss}.bmp'
        print(imgName,datetime.now())
        pxFlux=setMFETarget(imgName,power,TheMFE,opBng,ncol,nrow)
        optmSingle(TheSystem,parN,opBng,ncol,nrow,i)

The running results show as
p0000.bmp 2024-04-09 21:06:05.082130
optimization = 0 2024-04-09 21:06:12.615857
optimization = 1 2024-04-09 21:06:19.887321
optimization = 2 2024-04-09 21:06:29.702357
optimization = 3 2024-04-09 21:06:42.049635
optimization = 4 2024-04-09 21:06:52.835375
optimization = 5 2024-04-09 21:07:04.246019
optimization = 6 2024-04-09 21:07:17.676560
optimization = 7 2024-04-09 21:07:32.591460
optimization = 8 2024-04-09 21:07:52.054525
optimization = 9 2024-04-09 21:08:11.587495
optimization = 10 2024-04-09 21:08:42.282695
optimization = 11 2024-04-09 21:09:17.221488
optimization = 12 2024-04-09 21:09:39.321940
optimization = 13 2024-04-09 21:10:01.000962
optimization = 14 2024-04-09 21:10:32.857089
optimization = 15 2024-04-09 21:10:59.061922
Variable Fixed = 15 2024-04-09 21:11:37.625550
Over 1 OPMT = 15 2024-04-09 21:11:37.752509

p0001.bmp 2024-04-09 21:11:38.052760
optimization = 0 2024-04-09 21:11:45.741525
optimization = 1 2024-04-09 21:11:52.989084
optimization = 2 2024-04-09 21:12:03.875797
optimization = 3 2024-04-09 21:12:15.174620
optimization = 4 2024-04-09 21:12:27.627474
optimization = 5 2024-04-09 21:12:45.601644
optimization = 6 2024-04-09 21:13:08.888544
optimization = 7 2024-04-09 21:13:33.687291
optimization = 8 2024-04-09 21:14:06.230512
optimization = 9 2024-04-09 21:14:36.535561
optimization = 10 2024-04-09 21:15:16.283523
optimization = 11 2024-04-09 21:16:05.355903
optimization = 12 2024-04-09 21:16:45.097957
optimization = 13 2024-04-09 21:17:39.770961
optimization = 14 2024-04-09 21:19:03.867097
optimization = 15 2024-04-09 21:20:30.030670
Variable Fixed = 15 2024-04-09 21:22:02.387894
Over 1 OPMT = 15 2024-04-09 21:22:02.522653

p0002.bmp 2024-04-09 21:22:02.831023
optimization = 0 2024-04-09 21:22:18.315095
optimization = 1 2024-04-09 21:22:40.599438
optimization = 2 2024-04-09 21:25:36.438176
optimization = 3 2024-04-09 21:31:50.944993
optimization = 4 2024-04-09 23:23:54.578955
optimization = 5 2024-04-10 00:23:31.163572
optimization = 6 2024-04-10 01:57:20.572472
optimization = 7 2024-04-10 03:29:10.087029
optimization = 8 2024-04-10 05:08:59.642493
optimization = 9 2024-04-10 06:55:14.068808


For p0002.bmp, the optimization is too slow. 
Does any method be able to speed up the optimization?

jwbeenakker
Userlevel 3

What happens when you start the optimisation with p0002.bmp?

Cheng-Mu Tsai

Hi, jwbeenakker
Just same as the p0000.bmp. 
step 1 According to the gray level of photo p0002.bmp (10x26 pixels), 
          set the MFE targe from the operator 7 to operator 266.
step 2 Get random value to the radiuses of biconic Zernike surface. 
step 3 Then run optmSingle(TheSystem,parN,opBng,ncol,nrow,i) 
But the p0002.bmp is same as p0000.bmp and p0001.bmp. 
 

David.Nguyen
Rank
Userlevel 7
Badge +2

@Cheng-Mu Tsai

 

Sorry to insist but could you try opening and closing the optimizer only once? I just want to see if it makes a difference.

def optimization(LocalOpt):
    #Automatic=0, Fixed_1_Cycle=1, Fixed_5_Cycles=2, 
    #Fixed_10_Cycles=3, Fixed_50_Cycles=4, Infinite=5  
    LocalOpt.Cycles = ZOSAPI.Tools.Optimization.OptimizationCycles.Fixed_5_Cycles
    LocalOpt.RunAndWaitForCompletion() 


def optmSingle(..., LocalOpt):
    ...
    optimization(LocalOpt)


if __name__ == '__main__':
    LocalOpt = TheSystem.Tools.OpenLocalOptimization() 

    for i in range(3):
        ...
        optmSingle(..., LocalOpt)

    LocalOpt.Close()

Take care,

 

David

Cheng-Mu Tsai

Hi, David.Nguyen,
Thanks for your suggestion. 
The suggestion program seems to work normally. 
But I doubt that the problem comes from the object (Biconic Zernike Surface) for Sag problem.
When this object sets the X-Half-Width or Y Half-Width to be larger than Radius X or Radius Y,
it could show error as 


Does the ZOS-API support any exceptions when error happens such as sag problem?

Does the ZOS-API support the optimization escape when the running too much time?

David.Nguyen
Rank
Userlevel 7
Badge +2

@Cheng-Mu Tsai 

 

Apologies, I am not saying the problem is related to a specific object. I was wondering if the problem is that when you start the optimization the first time, some cores get reserved for the optimization and even though you close the optimizer, perhaps they could still be locked somehow? This means you have less cores for the second optimization, and eventually you run out of cores? Again, I’m not saying this is the problem, but I’m just curious to know whether it makes a difference.

Take care,


David

Cheng-Mu Tsai

Hi, David.Nguyen, 

Does the ZOS-API support any exceptions or optimization interrupt when error happens such as sag problem or optimization too much time?

David.Nguyen
Rank
Userlevel 7
Badge +2

@Cheng-Mu Tsai 

 

I don’t think an exception is raised for an error message like you are showing and I also don’t think it has a timeout that would prevent long optimization times. I could be wrong though. You may want to have a read at this post:

Take care,

 

David

Reply


Community Posting GuidelinesCookie settings


Zemax Community

Need more help?

To Chinese users:

Do not provide any information or data that is restricted by applicable law, including by the People’s Republic of China’s Cybersecurity and Data Security Laws ( e.g., Important Data, National Core Data, etc.).


不要提供任何受适用法律，包括中华人民共和国的网络安全和数据安全法限制的信息或数据（如重要数据、国家核心数据等）。



Privacy policy   License Agreements    Terms of use    Cookie Policy   Zemax Account Licensing Policies