3D Tympanoscope
UPDATE (29/07/2020): A European patent application ‘METHOD AND APPARATUS FOR OBTAINING A 3D MAP OF AN EARDRUM’ was published on July 29, 2020 under publication n° EP3685729.
https://worldwide.espacenet.com/patent/search/family/065200732/publication/EP3685729A1?q=%20EP%203685729
Contact: Sam Van der Jeught (sam.vanderjeught@uantwerpen.be), Matthias Dillen (matthias.dillen@uantwerpen.be), Joris Dirckx (joris.dirckx@uantwerpen.be)
Research
The tympanic membrane (TM) or eardrum is a conically shaped thin membrane which separates the outer ear from the middle ear. It conducts sound vibrations from the external ear canal to the ossicles and protects the middle ear from infections. The three-dimensional shape of the TM plays a crucial role in this process and any structural change to its topography is an important indicator for existing or impending pathology or hearing impairment.
In order to link the pathophysiological mechanisms of chronic middle ear disease (such as retraction pockets and cholesteatoma) to loss of TM stiffness, von Unge et al. have conducted a series of ex vivo tympano-topography experiments on gerbil ears [2]. By measuring the 3D pressure-induced shape deformations of TMs, with varying degrees of otitis media (inflammation of the middle ear) when increasing levels of pressure were applied over them, a correlation between TM volume displacement and stage of disease could be made.
Therefore, as aberrant local deformations or ‘weak spots’, due to loss of stiffness of the TM, could indicate structural damage and be an important precursor for cholesteatoma formation and ultimately hearing loss, a need for quantitative evaluation of the TM shape in-vivo has been expressed. A technique for in-vivo measurement, as needed in clinical diagnosis, is however still missing.
Based on this conclusion, the idea of developing a tool, for quantitative evaluation of the TM shape, came to life. Then, further research was required in order to analyze the various techniques which are usable for measuring the 3D-shape of the tympanic membrane.
Inspiration for the use of structured light profilometry, to obtain 3D-shape measurements of the tympanic membrane, was taken from the research of Zhang et al. [3] They proposed a combination of structured light profilometry and graphics processing unit programming. Although these setups obtain high-resolution (≈ 5-10% of projected fringe period) depth scans of the measurement target surface, these setups are typically designed for macroscopic objects only and acquiring high-speed and high-resolution depth information of microscopic objects currently remains challenging.
Figure 1: (I) ‘Zero-generation’ otoscopic unit and (II) schematic overview of its optical layout with enlarged detail of otoscope head. DMD: digital micromirror device, PL: projection lens, SM: semi-transparent mirror, OL: object lens.
In recent work [1], we have developed a custom handheld device based on a modified clinical otoscope that simultaneously projects structured light patterns onto the eardrum and records them with a digital camera, placed at a relative angle to the projection axis (see Figure 1). By employing a high-speed camera and by using parallel programming techniques, we have demonstrated that 3D shape data of cadaveric human eardrums can be obtained at real-time video rates, allowing the indirect monitoring of dynamic middle ear cavity pressure fluctuations. However, due to the relatively high transparency of eardrum tissue, the contrast of the reflected fringe patterns is insufficiently high to extract high-resolution depth data from the topograms without first coating the membranes with white cosmetic dye. This currently negates the noninvasive nature of our technique and complicates its application in the clinical setup on living patients.
Development
Although the first prototype worked fine on ex-vivo test subjects, there were still several aspects that required a redesign before in-vivo testing could be initiated. Redesigning this prototype led us to the second and more usable prototype. This prototype used a new optical setup, which features a new projection unit and camera. By using only a single projection grid, the overall weight and size of the prototype could be reduced.
Figure 2: (I) Second generation of the otoscopic unit and (II) schematic overview of its optical layout.
Definition
The 3D-Tympanoscope is a compact medical device that will be used by ENT-physicians in the clinical office. By measuring the volumetric displacement of the human tympanic membrane as a result of middle ear cavity pressurization, a variety of existing or impending middle ear pathologies can be detected.
The device contains a new single-shot optical engine to observe the surface profile of the eardrum and to display its 3D-shape in real-time. This non-invasive diagnostic tool will assist the ENT-physician to diagnose specific conditions, such as malfunctioning Eustachian tube and weak areas of the eardrum, more easily and accurately.
Problem analysis
To date, visual examination of the eardrum is performed using a classic otoscope. This device helps doctors to observe pathologies such as perforations and diagnose damage or inflammation of the tympanic membrane.
However, the information provided by the otoscope remains limited to a flat 2D image. This image does not lead to any understanding about the shape of the tympanic membrane, which help when identifying other pathologies such as a malfunctioning Eustachian tube or weak zones in the eardrum.
Key features
The technology
The device employs a structured light profilometry technique in which a sinusoidally modulated light pattern is projected onto the eardrum and observed by a digital camera, placed under a relative angle to the projection axis.
The recorded images are processed in real-time using a novel deep learning-based algorithm to extract the full-field 3D deformation data.
Figure 3: (I) The image captured by the small camera unit. The projected lines are visible on the observed object. (II) A height map is generated based of the projected pattern, which is deformed by the shape of the object.
Mobility
Both the design of the device and the controls are kept simple in order to ensure an efficient use of the device, regardless of the situation of usage. Since the device is connected to the computer via a simple USB connection and light is passed through an optical fiber, the doctor is able to use the device at a significant distance from the processing unit (computer).
Flexibility
Since the length of the ear canal can vary between patients and the length of the speculum depends on the characteristics of the manufacturer, a simple adjustment system needs to be implemented into the device. This system offers the physician the possibility to reposition the focus point of both the camera module and the sinusoidal grid pattern, which results in better 3D-images of the tympanic membrane.
[1] Van der Jeught, S., Dirckx, J. J. (2017). Real-time structured light-based otoscopy for quantitative measurement of eardrum deformation. Journal of biomedical optics, 22(1); 016008.
[2] Shape and displacement patterns of the gerbil tympanic membrane in experimental otitis media with effusion, von Unge M., Decraemer W.F., Dirckx J.J., and Bagger-Sjöbäck D., Hearing Research 82 (2), pp 184-196 (1995).
[3] High-resolution, real-time three-dimensional shape measurement on graphics processing unit, Karpinsky, N., Hoke, M., Chen, V., & Zhang, S., Optical Engineering 53(2), 024105 (2014).