Solved

ZRDLoader in Python?

  • 9 February 2020
  • 13 replies
  • 236 views

  • Single Emitter
  • 3 replies
Is it possible to use the ReadZRDData etc. functions of the RayTrace.dll or ZRDLoaderFull.dll in Python? How?
icon

Best answer by Ivo 15 February 2020, 14:09

View original

13 replies

Userlevel 6
Badge +2
Yes it is possible to use those functions in Python, but rather than using the pywin32 module to connect Python to the ZOS-API (which is the current method for connecting Python to the ZOS-API), you will need to use the pythonnet module to make the connection.

To use a C# DLL in Python, ensure you are running Python 3.4, 3.5, 3.6, or 3.7 as PythonNET does not work with Python 3.8 yet.



To check that pythonnet is installed, you can type the following lines in a command window:

python -m pip install pythonnet

If it is not installed, it will install it.



Then I have attached 2 new templates that use pythonnet to connect to the ZOS-API. As you will see, those templates import the clr module which is needed :

import clr

crl.AddReference()




We are working on fully supporting pythonnet in our next releases. The templates will be provided using that technology. Codes using the pywin32 module however will still work.
Thanks!

I succeeded in installing pythonnet and running your PythonNETZOSConnection.py. But I still can't load the RayTrace.dll or ZRDLoaderFull.dll. I am not an experienced programmer - neither python nor .net - hence I'd be very happy to get another hint.



I tried the following:

...
Loader = os.path.join(os.sep, r'C:\Users\...\ZRDLoaderFull.dll')
print('File exists:', os.path.isfile(Loader))
clr.AddReference(Loader)

And I am getting:

(from your PythonNETZOSConnection.py)
Found OpticStudio at: %sc:\program files\zemax opticstudio
Connected to OpticStudio
Serial #: XXXXX (my SN)
(from my lines)
File exists: True
Traceback (most recent call last):
... in <module>
clr.AddReference(Loader)
FileNotFoundException: Unable to find assembly 'C:\Users\...\ZRDLoaderFull.dll'.
bei Python.Runtime.CLRModule.AddReference(String name)

Same with RayTrace.dll

From the web I learned "Unable to find assembly ..." may also mean "Unable to load assembly ..." 

Environment is:

Win10(64bit), Anaconda3(64bit), Spyder 4.0.1, Python 3.7.6, IPython 7.11.1

pythonnet 2.4.0 py37_1 conda-forge

Zemax OpticStudio 20.1 Professional (64-bit)

 
Userlevel 6
Badge +2
Your code looks correct to me.

Could you try this?

clr.FindAssembly(assemblypath)

Otherwise I have attached an example that works for me. I have recompiled the dll and copied it under {Zemax}\ZOS-API\Libraries.

Let us know if you manage to fix the issue or not.
Nope. Same error messages as before. With your file as well. Probably some very system specific nicety (or oddity in my particular setup).Tried downgrading to python 3.6.10, pythonnet 2.3.0. Same as before.



clr.FindAssembly(assemblypath) has no output whether there is a fileassemblypath or not.
Userlevel 6
Badge +2
Hi Ivo



Sorry it didn't work. I have discussed with one of my colleague and could you try the following steps:

-Download the raytrace.dll from that article. It is the official version: Batch Processing of Ray Trace Data using ZOS-API in MATLAB: https://my.zemax.com/en-US/Knowledge-Base/kb-article/?ka=KA-01651

- Copy the attached "PythonNET - ZRDLoaderFull.py" file to a folder.

- Copy the DLL to the same folder.

- Then run the Python code.



If it doesn't work, check if you have a file called BatchRayTrace.py on your computer. I had one and it prevented it from working. If you have one, rename it.



Let us know how it goes.



Sandrine
Hi Sandrine,



I think I found the problem and a simple solution. But I still don't know why it is a problem. Maybe a total beginners mistake 😞 and everybody else knows and avoids this. Maybe some security issue specific to our companies systems in order to avoid corrupted files being executed? Anyway I'd like to apologize for causing you unnecessary work.



When I downloaded the zips and extracted the files with Windows Explorer the modification date of the RayTrace.dll got set to the current date, which is apparently not OK, despite of everything else being identical according to CRC and SHA checksums. Extracting files with 7-zip the original dates are kept and no problem occurs.



Thanks again for your efforts and fast responses! I appreciate that very much!

I think I'll like the pythonnet connection much better then the pywin32 version :-)



Best regards

Ivo



 

Hi, 


I've downloaded the files and tried to use the dll. The dll itself works, but for my simulation needs, I don't need to read all the segments, just the segments where the ray undergoes wavelength shift, and the next one after that. ReadNextSegmentFull() provides a field with the info if the ray has undergone wavelength shift. Is this not the case for this dll?


It would be so nice to use the dll, everything goes remarkably faster.


Thank you.

Userlevel 4
Badge +1

Hi Elisavet,


I'm sorry about the delay. I got this answer from a senior developer here, and am passing it on to you and the Zemax community verbatim:


 


You cannot easily pass enum values like RayStatus in the DLL (at least the way it was originally setup). In the DLL, the RayStatus, which has the flag for WaveShifted, is converted to an bit-wise integer where all the statuses are consolidated to a single value. To “decompose” the integer into the string value for the RayStatus, you can cast the integer back to an enum. Here is an excerpt from doing this conversion for Matlab, but the process will be identical for doing it in PythonNET:


 


The Status value comes from the IZRDReaderResults interface and the RayStatus enum. The enum itself is stored as a bitwise signed integer and the actual logic to summarized the RayStatus enum uses a bitwise shift operation to “sum” the integers together to get a “final” status. Since Matlab doesn’t handle enums well, the enum itself is cast to it’s integer value to pass back to Matlab. If the RayStatus is stored in the variable status, then you can use the following code to convert from the integer to the RayStatus:


 


RayStatusEnum = ZOSAPI.Tools.RayTrace.RayStatus;


System.Enum.ToObject(RayStatusEnum.GetType, status(5))


 


If status(5) has an integer of 38, then you will get the following output:


 


Reflected, Transmitted, GhostedFrom

I use this in Python. I can see if a segment is downconverted by the status field. Although the value is supposed to be an enum, for me, I was told this was because of the COM link, this value is 1653 or sth like that, I am not in the office right now and cannot check the value. I didn't get what your answer answers, sorry. 


The point is that after I run the dll, all segments of all rays are saved serially, eg for the status field of the dll - you get a big numpy array with predefined size, and although it's fast to read the segments, I have my doubts if the overall time to manipulate the data afterwards would be so fast. I will have to loop at some point, I don't know if all could be solved with vectorization in my case. So, the dll would miss its purpose.


I asked if I could selectively read ray segments based on the status value of each segment for example.

Userlevel 5
Badge +1

Hi Elisavet,


Thanks for your follow-up here!


After speaking about your comments with someone from our Dev team, I think we can provide some clarifications. First off, you are right -- running the DLL will have all segments of the ray saved serially. However, while Python does have some inefficiencies in looping thorough large structures of data (hence the creation of the DLL in the first place), the usage of Numpy itself will also leverage some other compiled C++ code. So, while there might be some benefit to having a wholly-contained DLL to read through and filter out the data you're interested in (ray segments undergoing downconversion), the difference between that and using the numpy array probably won't be that noticable due to the processing offload done by numpy.


Again, this provided DLL won't be able to do filtering from within the execution -- that would have to be written and compiled on your end for that kind of functionality. However, given how the read data is structured in Python, we think there should be a benefit in the execution of this ray reading procedure with the DLL and numpy library.


Please let us know if you have any further questions or if I've misunderstood you in any way! Thanks again for your time here.


~ Angel

Badge +1

You can filter on the RayStatus either natively in Python or using Numpy by using a bitwise 'and' operation between the status and the enum value of the given status.  The RayStatus field is a bitwise operator to consolidate all the different statuses for a single ray segment.  Performing a bitand operation between the desired enum and the RayStatus will allow you to determine if the given ray segement's status meets that enum value.  If the bitand is greater than 0, the ray meets that status.  For the ReadNextSegment() via the ZOS-API directly, the code would look like:


rays = tool_results.ReadNextResult(1, 2, 3, 4)

while rays[0]:

    segments = tool_results.ReadNextSegment(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20)

    idx = 0

    while segments[0]:

        status = segments[6];

            

        # perform bitand to check raystatus

        if status & ZOSAPI.Tools.RayTrace.RayStatus.WaveShifted > 0:

            print('Segment %2i has a wavelength shift' % idx)

                

        idx += 1

        segments = tool_results.ReadNextSegment(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20)


    rays = tool_results.ReadNextResult(1, 2, 3, 4)


You can get the same check with Numpy using the bitwise_and method (functions and variable names from the previously attached PythonNET - ZRDLoaderFull.py example):


totalStatus = zos.LongToNumpy(zrdData.Status)

waveShiftedArray = np.bitwise_and(ZOSAPI.Tools.RayTrace.RayStatus.WaveShifted, totalStatus) > 0

For the above issue, I resolved it the following way: I read the statusData within the loop and got the indices of the downconverted rays with an np.where(statusData==1536). I didn't need to loop over anything. I used np.add.at() to calculate the total absorbed flux after binning the rays in self-made voxel grid. The way I am doing the binning is pretty simple, and perhaps I need to rethink how to do this. 


A very important think to note, is that in order to process all the rays outside the loop you need to increase the number of segments to the actual number of segments, that's roughly 30*no_rays, otherwise you lose info.


I would have posted the code, but don't see a button for code insertion. It's been somewhat frustrating.


 


Another problem I am having now is that this is using NET, and I don't have access to COM constants. I have written a class using this BatchRayTrace, and I want to write a function that changes the source type. Eg. from Gaussian to Laser Diode, or source Ellipse. How can I do that?


What I've found is this example:


TheNCE = TheSystem.NCE

Object_1 = TheNCE.InsertNewObjectAt(1)


oType_1 = Object_1.GetObjectTypeSettings(constants.ObjectType_SourcePoint)

Object_1.ChangeType(oType_1)


But now I don't have access to constants, because there isn't a COM connection, so that dictionary is not initialised, and I get an attribute error. 


 


Another point: I can access things from ObjectData in this regime


eg. TheSystem.NCE.GetObjectAt(self.source_no).ObjectData.NumberOfAnalysisRays


Things that are not listed in examples are impossible to find. Can someone explain how the documentation works? I search NCE ObjectData, I get a banch of results. Ok, there are some examples I can extract fields from, but how can I find what I can access through a single tag? I am trying to change the power of the source and cannot find the field through ObjectData.


Edit: Found it, but it got way longer than it should


Another off topic question: Is there a way to return a string or sth to find out what type the object is? Eg. Source Gaussian or source ellipse?


ps. I am using Python

Userlevel 6
Badge +2

Hi Elisavet,


Yes - one of the differences between Python COM and .NET is that the enumerated variables are no longer located in the 'constants' library. Actually, I have discussed this in another post here: What is the difference between Python COM and Python .NET? In that post, I showed how to find the full title for an enumerated variable. In your case, you can replace constants.ObjectType_SourcePoint with ZOSAPI.Editors.NCE.ObjectType.SourcePoint. This is exemplified in sample file #17 within the ZOS-API/Sample Code folder.


Regarding your quesiton about the documentation, we have a guide on that here: Navigating the ZOS-API Syntax Help document - Part 1. We also have some additional discussion on object-oriented programming here: Understanding the basics of ZOS-API structure


Finally - yes, there is a way to return a string with the object type. In general, items within the Non-Sequential Editor spreadsheet (such as the Type column) can be access through the INCERow Interface. Within that interface, we see the following properties are available:


 



 


All of these are 'get'-type. In other words, they represent data that may be extracted and printed. The TypeName property will report data as a string. So, to report an object's type in the way you are looking for, we can use that. Here is an example:


 


TheNCE = TheSystem.NCE

obj1 = TheNCE.GetObjectAt(1)

type = obj1.TypeName

print(type)

 


Let us know if you have any other questions about this!


Best,


Allie


PS: I typically paste code with the 'Special Container' or 'Computer Code' style:


 



 


You are also welcome to attach code snippets via the Attach a file dialog! The website will accept *.py files!


 

Reply