Forming and Displaying Video Data in Onboard Enhanced Vision Multispectral Systems

Abstract

The article considers the results of long-term research and real flight tests for developing a multichannel system for enhanced vision in difficult for aircraft crew visibility conditions. The multicomponent nature of the aggregate video signal generated by a multispectral video system is shown, justifying its representation in color and the need to develop a color indicator of the aircraft windshield to obtain a better quality of situational awareness for pilot compared to the capabilities of traditionally used monochrome indicators. A method for integrating monochrome images of multichannel enhanced vision system spectral channels into a color multicomponent image is proposed. For more natural perception of the resulting color images by the human eye, the spectrum of the spectral characteristics of the video channels is rearranged in the direction of decreasing wavelength. For the most common RGB color image format, the red component R of the display system records the image of the long-wave thermal range, the green component G records the image of the mid-wave infrared range, the blue component B records the image of the short-wave visible television range. The resulting pseudo-color image allows unambiguous restoring the original images. This proves the absence of information loss during image integration. Since there are no colored windscreen displays yet, a method for reproducing a pseudo-color image on a mono-chrome display is proposed. To convert a pseudo-color image to monochrome one, methods are used in which color contrasts are preserved in the form of tonal contrasts. The advantages of the proposed technology are demonstrated on real examples

Please cite this article in English as:

Bondarenko M.A., Bondarenko A.V., Drynkin V.N., et al. Forming and displaying video data in onboard enhanced vision multispectral systems. Herald of the Bauman Moscow State Technical University, Series Instrument Engineering, 2022, no. 2 (139), pp. 79--95 (in Russ.). DOI: https://doi.org/10.18698/0236-3933-2022-2-79-95

References

[1] Vizil’ter Yu.V., Zheltov S.Yu. [Technical vision problems in modern aviation systems]. Tr. nauch.-tekh. konf.-sem. "Tekhnicheskoe zrenie v sistemakh upravleniya mobil

[2] Akopdzhanyan Yu.A., Bilanchuk V.V., Drynkin V.N., et al. Flight experimental research of on-board multispectral imaging system for all-weather aircraft landing. Herald of the Bauman Moscow State Technical University, Series Instrument Engineering, 2014, no. 5 (98), pp. 60--68 (in Russ.).

[3] Bondarenko A.V., Bondarenko M.A. Hardware-software solution of multispectral enhanced vision system. Sovremennaya elektronika, 2017, no. 1, pp. 32--37 (in Russ.).

[4] Bondarenko M.A., Bondarenko A.V., Yadchuk K.A., et al. Sposob sovmeshcheniya odnovremenno poluchaemykh izobrazheniy ot matrichnykh fotopriemnikov raznogo spektral’nogo diapazona [Method of combining simultaneously obtained images from matrix photodetectors of different spectral range]. Patent RU 2705423. Appl. 17.01.2019, publ. 07.11.2019 (in Russ.).

[5] Rukovodstvo R-315. Po minimal’nym standartam kharakteristik aviatsionnykh sistem (MASPS) dlya sistem uluchshennogo videniya, sistem II, kombinirovannykh sistem iskusstvennogo videniya i bortovykh sistem uvelicheniya dal’nosti videniya [RTCA DO-315. Minimum aviation system performance standard (MASPS) for enhanced vision systems, synthetic vision systems, combine vision systems and enhanced flight vision systems]. Moscow, ARMAK, 2012.

[6] Bondarenko M.A., Drynkin V.N., Nabokov S.A., et al. Adaptive algorithm for selecting informative channels in on-board multispectral video systems. Programmnye sistemy i vychislitel’nye metody [Software Systems and Computational Methods], 2017, no. 1, pp. 46--52 (in Russ.). DOI: http://doi.org/10.7256/2454-0714.2017.1.21952

[7] Bondarenko M.A., Bondarenko A.V. Image forming in multispectral video systems for visual and automatic nondestructive testing. Uspekhi prikladnoy fiziki [Advances in Applied Physics], 2018, vol. 6, no. 4, pp. 325--332 (in Russ.).

[8] Bondarenko M.A., Drynkin V.N. Image informative increasing for onboard enhanced vision systems. Tekhnicheskoe zrenie, 2015, no. 1, pp. 57--65 (in Russ.).

[9] Lebedev M.A., Komarov D.V., Vygolov O.V., et al. [Images complexation in different spectral bands based on FusionNet adversarial generative neural network]. Sb. tez. dokl. Vseros. nauch.-tekh. konf. "Tekhnicheskoe zrenie v sistemakh upravleniya" [Abs. Russ. Sc.-Tech. Conf. Technical Vision in Control Systems]. Moscow, IKI RAS Publ., 2019, pp. 48--49 (in Russ.).

[10] Kolodiychuk A. Top-5 nedorogikh proektorov na lobovoe steklo avto s AliExpress [Top-5 of cheap car wind screen projectors from AliExpress]. bycars.ru: website (in Russ.). Available at: https://bycars.ru/journal/top-5-luchshih-proektorov-na-lobovoe-steklo-avto-s-aliexpress_3345 (accessed: 13.05.2021).

[11] DLP5531-Q1 0.55-inch 1.3-megapixel DMD for automotive exterior lighting. ti.com: веб-сайт. URL: https://www.ti.com/document-viewer/DLP5531-Q1/datasheet/features-dlps0001089#DLPS0001089 (дата обращения: 31.03.2021).

[12] Teplovizionnyy modul’ FLM1280-SDI (LWIR) [FLM1280-SDI (LWIR) thermal imaging module]. npk-photonica.ru: website (in Russ.).Available at: https://www.npk-photonica.ru/product/21627 (accessed: 01.04.2021).

[13] Fomin F.V. Sovremennoe sostoyanie i perspektivy razvitiya zarubezhnykh IK-sistem [Current state of the art and development prospects for foreign IR-systems]. Moscow, MNITI Publ., 2018.

[14] Small, high performance InGaAs camera module with high image resolution. xenics.com: website. Available at: https://www.xenics.com/short-wave-infrared-imagers/xsw-640-series (accessed: 01.04.2021).

[15] Kak pokorit’ monokhromnyy rezhim? [How to deal with monochrome mode?] fotogora.ru: website (in Russ.). Available at: https://fotogora.ru/kak-pokorit-monokhromnyy-rezhim (accessed: 15.04.2021).

[16] Kak perevesti tsvetnoe izobrazhenie v cherno-beloe [How to transverse a chromatic image into black-and-white one]. livejournal.com: website (in Russ.). Available at: https://fotomeridian.livejournal.com/25340.html (accessed: 15.04.2021).

[17] Tsvetovaya sistema PAL [PAL colour system]. hisour.com: website (in Russ.). Available at: https://www.hisour.com/ru/pal-color-system-25921 (accessed: 21.04.2021).