Skip to main content
Solved

Custom Operands with Python

  • September 27, 2023
  • 3 replies
  • 420 views
cameron.vo

Hello. I have been learning how to use ZOS-API in Standalone mode with Python for a while now, but it seems that custom operands for optimization are only available using C# or C++. I have started working with C# (despite not knowing the language) and I have managed to get custom operands to work using this. Currently, I am working on connecting C# to Python so I can run my previously written scripts and then optimize the system based on some values calculated in Python. My current pipeline looks something like

  1. UDOC operand runs Python script
  2. Python script runs an instance of Zemax in Standalone mode and calculates some value
  3. Value is sent back to UDOC operand
  4. Zemax retrieves value

This setup somehow manages to work with just getting the value from Python into Zemax in the Merit Function Editor, but actually using the UDOC operand for optimization doesn’t seem to work and is also incredibly slow. A single cycle in the optimization loop can take over two minutes while retrieving the UDOC operand value in the Merit Function Editor only takes a couple of seconds, so I assume the main problem with this setup has something to do with the interaction between Zemax and Standalone Mode being looped due to the nature of optimization. My main question regarding this topic is whether or not there are other methods to creating custom operands using Python. Based on what I have done so far, I currently think that it is probably best to just convert most or even all of my current Python code into C#, which will be a daunting task as my code is quite complex. I have also considered making my own optimization algorithm in Python which would completely circumvent this issue. I want to hear some opinions regarding this matter before doing anything.

Best answer by MichaelH

Hey Cameron,

There are a few things to consider with your workflow:

  • For the UDOC, make sure you have the EXE compiled as a Windows Application and not a Console Application.  Each time the MF updates, the UDOC is executed and GUI I/O such as displaying the command window is expensive.
  • Launching a Standalone Application via Python is probably taking the most time for each cycle.  The typical connection between a Standalone Application and the ZOS-API is 10’s of seconds.  
  • Depending on how you’re communicating between C# & Python, you could run into a handful of issues:
    • If you’re calling python via exec(), then Python is launched on a new thread and there is no communication between your C# UDOC and Python.  You’ll have to “guess” at when Python is done, most likely by querying an output file from Python (this is an expensive operation)
    • If you’re using IronPython, then you’re limited in the set of Python modules you can use within your script.  No modules with compiled code (such as numpy or matplotlib) can be used with PythonNet (I suggest MathNET for numeric calculations and ScottPlot for graphing, both available via nuget).  The advantage of IronPython is you don’t need Python installed on the target machine.
    • If you’re using PythonNET and with using (Py.GIL()), you lose all multi-threading capability.  Although the MF sequentially updates each operand individually, a single operand can be evaluated with multiple threads.

So, although it is a “cool experiment” to get Python running from C#, you will never see the performance you’re expecting with a UDOC.  I would suggest converting all your code to C#.  Another benefit of the UDOC is if it’s compiled correctly, you only need 1 EXE file and this 1 file is stored in any ZAR if the UDOC is loaded in the MF.  This makes a pure C# UDOC extremely transferable between users.

Also, this is a great time to remind Zemax Engineers and Developers that UDOCs are still called an a executable and not on the same process as OpticStudio itself.  This makes UDOCs much slower than built-in operands.  Zemax should implement in-process UDOCs to really help expand the ZOS-API’s capabilities. 

View original
    Did this topic help you find an answer to your question?

    3 replies

    jwbeenakker
    • jwbeenakker
    • Visible
    • 28 replies
    • September 27, 2023

    Hi Cameron,

    Kudos for implementing that in C# 😀

    In principle there should be no difference in the parts of Zemax that you can set via python or C# (they use the same  API). In ZOSPy we have included some basic functionality with the Merit Functions, but probably not something as sophisticated as you require.  This is something that we aim to expand in the future (also with some examples), but as it is currently not critical to our scientific work, I don’t have a timeline on that.

    On the other hand, if you want/can share some of your efforts, we are happy to include it in a future release of ZOSPy. In that case, it is probably easier to start an discussion or pull-request on the ZOSPy’s GitHub page, so we can jointly work on it.

    @Luc.van Vught or @chaasjes : do we have any of our merit function examples/code that we can easily share?

    Mark.Nicholson
    1 person likes this
    • Mark.Nicholson
      Mark.Nicholson
    MichaelH
    Ansys Staff
    Forum|alt.badge.img+2
    • MichaelH
    • Ansys Staff
    • 352 replies
    • Answer
    • September 27, 2023

    Hey Cameron,

    There are a few things to consider with your workflow:

    • For the UDOC, make sure you have the EXE compiled as a Windows Application and not a Console Application.  Each time the MF updates, the UDOC is executed and GUI I/O such as displaying the command window is expensive.
    • Launching a Standalone Application via Python is probably taking the most time for each cycle.  The typical connection between a Standalone Application and the ZOS-API is 10’s of seconds.  
    • Depending on how you’re communicating between C# & Python, you could run into a handful of issues:
      • If you’re calling python via exec(), then Python is launched on a new thread and there is no communication between your C# UDOC and Python.  You’ll have to “guess” at when Python is done, most likely by querying an output file from Python (this is an expensive operation)
      • If you’re using IronPython, then you’re limited in the set of Python modules you can use within your script.  No modules with compiled code (such as numpy or matplotlib) can be used with PythonNet (I suggest MathNET for numeric calculations and ScottPlot for graphing, both available via nuget).  The advantage of IronPython is you don’t need Python installed on the target machine.
      • If you’re using PythonNET and with using (Py.GIL()), you lose all multi-threading capability.  Although the MF sequentially updates each operand individually, a single operand can be evaluated with multiple threads.

    So, although it is a “cool experiment” to get Python running from C#, you will never see the performance you’re expecting with a UDOC.  I would suggest converting all your code to C#.  Another benefit of the UDOC is if it’s compiled correctly, you only need 1 EXE file and this 1 file is stored in any ZAR if the UDOC is loaded in the MF.  This makes a pure C# UDOC extremely transferable between users.

    Also, this is a great time to remind Zemax Engineers and Developers that UDOCs are still called an a executable and not on the same process as OpticStudio itself.  This makes UDOCs much slower than built-in operands.  Zemax should implement in-process UDOCs to really help expand the ZOS-API’s capabilities. 

    ~Michael
    jwbeenakker
    • jwbeenakker
    • Visible
    • 28 replies
    • September 28, 2023
    jwbeenakker wrote:

    In principle there should be no difference in the parts of Zemax that you can set via python or C# (they use the same  API). In ZOSPy we have included some basic functionality with the Merit Functions, but probably not something as sophisticated as you require.  This is something that we aim to expand in the future (also with some examples), but as it is currently not critical to our scientific work, I don’t have a timeline on that.

    Some correction after I had some further discussion with Luc and Corne.

    In line with the post of @MichaelH , this is a bit more complex than I originally anticipated. The merit function operands can be set via python/ZOSPy, but for the UDOC you indeed need a compiled executable, so for that part python is less optimal. 

    Reply


    Community Posting GuidelinesCookie settings

    Cookie policy

    We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

     
    Cookie settings


    Zemax Community

    Need more help?

    To Chinese users:

    Do not provide any information or data that is restricted by applicable law, including by the People’s Republic of China’s Cybersecurity and Data Security Laws ( e.g., Important Data, National Core Data, etc.).


    不要提供任何受适用法律，包括中华人民共和国的网络安全和数据安全法限制的信息或数据（如重要数据、国家核心数据等）。



    Privacy policy   License Agreements    Terms of use    Cookie Policy   Zemax Account Licensing Policies