Hi AminR,

You can check out our knowledge base article Methods for analyzing MTF in OpticStudio.There is a detailed introduction to the different MTFs

FFTMTF: which is based upon Fraunhofer diffraction theory. The method involves a Fast Fourier Transform of a grid of rays traced to the pupil (hence the name FFT MTF). The resulting MTF is the modulation as a function of spatial frequency for a sine wave object; although optionally the real, imaginary, phase, and square wave response is available.

Geometrical MTF:  It is based on a Fourier transform of the geometric spot data, and uses a Gaussian Quadrature sampling scheme.

Best Regards
Sean