I’m working on running multiple POP simulations and they each take about 30 minutes. I’m debating trying out the python package “threading” to try and run multiple runs at the same time in order to save some computing time. Are there going to be any limitations on the Zemax side of this equation?
Solved
Python Multithreading with POP
Best answer by chaasjes
Hi,
I tried something similar and judging from my experience, this is not impossible. It's not easy, either, and I doubt if using multiple threads will improve run times, since POP already seems to use multiple threads. Quoting from the documentation:
The physical optics propagation routines in OpticStudio are designed to work on multiple CPU computers
That being said, there are some limitations on both the OpticStudio side and the Python side:
- You can't connect to multiple OpticStudio instances from a single Python script. More information about this here:
So if you want to perform the analyses in multiple OpticStudio instances, you will need to use multiprocessing instead of threading. - Instead of using multiple OpticStudio instances, you can create multiple optical systems in a single OpticStudio instance, which is probably the easiest way to achieve what you want. I recommend to do this in a standalone application. If you want to easily switch between extension mode (for development / debugging) and standalone mode (for the real simulations), you can try ZOSPy, which lets you switch between these modes by changing only a single line of code.
- Each thread or process needs to have a separate optical system or OpticStudio instance attached to it. Using multiprocessing or threading, this is hard (but maybe not impossible); when I tried this, I soon ran into problems. I ended up using Ray instead, which has support for stateful workers (which can be either threads or processes). This allows you to attach an optical system to a thread, and run multiple jobs sequentially in the same optical system (so you don't have to reinitialize the connection, although you should of course take care of removing the old system before you start a new simulation).
Hopefully this helps and I'm curious to see if you succeeded in getting parallel computations in OpticStudio working!
Hi,
I tried something similar and judging from my experience, this is not impossible. It's not easy, either, and I doubt if using multiple threads will improve run times, since POP already seems to use multiple threads. Quoting from the documentation:
The physical optics propagation routines in OpticStudio are designed to work on multiple CPU computers
That being said, there are some limitations on both the OpticStudio side and the Python side:
- You can't connect to multiple OpticStudio instances from a single Python script. More information about this here:
So if you want to perform the analyses in multiple OpticStudio instances, you will need to use multiprocessing instead of threading. - Instead of using multiple OpticStudio instances, you can create multiple optical systems in a single OpticStudio instance, which is probably the easiest way to achieve what you want. I recommend to do this in a standalone application. If you want to easily switch between extension mode (for development / debugging) and standalone mode (for the real simulations), you can try ZOSPy, which lets you switch between these modes by changing only a single line of code.
- Each thread or process needs to have a separate optical system or OpticStudio instance attached to it. Using multiprocessing or threading, this is hard (but maybe not impossible); when I tried this, I soon ran into problems. I ended up using Ray instead, which has support for stateful workers (which can be either threads or processes). This allows you to attach an optical system to a thread, and run multiple jobs sequentially in the same optical system (so you don't have to reinitialize the connection, although you should of course take care of removing the old system before you start a new simulation).
Hopefully this helps and I'm curious to see if you succeeded in getting parallel computations in OpticStudio working!
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
Need more help?
To Chinese users:
Do not provide any information or data that is restricted by applicable law, including by the People’s Republic of China’s Cybersecurity and Data Security Laws ( e.g., Important Data, National Core Data, etc.).
不要提供任何受适用法律，包括中华人民共和国的网络安全和数据安全法限制的信息或数据（如重要数据、国家核心数据等）。