Journal "Software Engineering"
a journal on theoretical and applied science and technology
ISSN 2220-3397

Issue N7 2024 year

DOI: 10.17587/prin.15.340-351
Method of Forming and Algorithm of Saving a Photo Map of the Seafloor during its Photography by an Unmanned Underwater Vehicle
V. F. Filaretov, Professor, Chief Researcher, filaretov@inbox.ru, Sevastopol State University, Sevastopol, 299053, Russian Federation, E. Sh. Mursalimov, PhD, Leading Researcher, murs@dvo.ru, A. A. Timoshenko, Research Associate, officesuit@mail.ru, M. D. Ageev Institute of Marine Technology Problems FEB RAS, Vladivostok, 690091, Russian Federation, A. V. Zuev, PhD, Leading Researcher, alvzuev@yandex.ru, Sevastopol State University, Sevastopol, 299053, Russian Federation
Corresponding author: Eduard Sh. Mursalimov, PhD, Leading Researcher, M. D. Ageev Institute of Marine Technology Problems FEB RAS, Vladivostok, 690091, Russian Federation, E-mail: murs@dvo.ru
Received on March 27, 2024
Accepted on June 04, 2024

The paper proposes a new method and algorithm for creating a photo map of the seabed from photographs obtained from on-board photo and video systems of autonomous uninhabited underwater vehicles during fulfillment of their various missions. This photo map is formed from photographs of the seabed by sequentially overlapping them. At the same time, data on the position of the underwater vehicle received from the on-board navigation system at the time of receipt of each photograph is used to position each photo image on the generated photo map. And to rotate and scale these photographic images, data on the orientation of the underwater vehicle in the global coordinate system and its height above the bottom, also obtained from the on-board navigation system, are used. To provide quick access to the photo map, it is proposed to store it on an on-board data storage device in the form of a tile map, widely used when creating interactive geographical maps, for example, Google Maps. However, the formation of a tile map is usually carried out on the basis of all the already received photographic images, as is done with satellite images, which cannot be applied on board underwater vehicles when data from the photo-video system is received sequentially. Therefore, a special algorithm has been developed and implemented that generates a tile map during the movement of the underwater vehicle and updates only the area of the tile map in which the photo is currently being taken. This minimizes the computational load and ensures the formation of a photo map of the bottom using standard on-board computing devices. The conducted semi-natural experiments with data obtained as a result of performing real missions using the MMT-3000 underwater vehicle confirmed the high efficiency of using the developed method and algorithm.

Keywords: autonomous uninhabited underwater vehicle, photo map, tile map, on-board photo and video system, navigation data, photo image processing
pp. 340—351
For citation:
Filaretov V. F., Mursalimov E. Sh., Timoshenko A. A., Zuev A. V. Method of Forming and Algorithm of Saving a Photo Map of the Seafloor during its Photography by an Unmanned Underwater Vehicle, Programmnaya Ingeneria, 2024, vol. 15, no. 7, pp. 340—351. DOI: 10.17587/prin.340-351. (in Russian).
This work was supported by the Sevastopol State University, project no. 42-01-09/253/2023-1.
References:
  1. Johansson B., Siesjo J., Furuholmen M. Seaeye Sabertooth, a hybrid AUV/ROV offshore system, Proceeding of the OCEANS 2010 MTS/IEEE SEATTLE, Seattle, WA, USA, 2010, pp. 1—3. DOI: 10.1109/OCEANS.2010.5663842.
  2. Albiez J., Joyeux S., Gaudig C. et al. Flatfish — a compact subsea-resident inspection AUV, Proceeding of the OCEANS 2015-MTS/IEEE, Washington, Washington, DC, USA, 2015, pp. 1—8. DOI: 10.23919/OCEANS.2015.7404442.
  3. Teledyne Marine: Gavia AUV software and data handling, available at: http://www.teledynemarine.com/gavia-auv-software-and-data-handling?BrandID=9
  4. Gornak V. S., Inzartsev A. V., Lvov O. Yu., Matvienko Yu.V., Scherbatyuk A. Ph. ММТ-3000 — a new small-sized autonomous uninhabited underwater vehicle of the Institute of Marine Technol­ogy Problems of the FEB RAS, Podvodnie Issledovaniya i Robototehnika, 2007, no. 1, pp. 12—20 (in Russian).
  5. Danckaers A., Seto M. L. Transmission of images by un­manned underwater vehicles, Autonomous Robots, 2020, vol. 44, no. 1, pp. 3—24. DOI: 10.1007/s10514-019-09866-z.
  6. Ahn J., Yasukawa S., Sonoda T., Nishida Y., Ishii K., Ura T. An optical image transmission system for deep sea creature sampling missions using autonomous underwater vehicle, IEEE Journal of Oceanic Engineering, 2018, vol. 45, no. 2, pp. 350—361. DOI: 10.1109/JOE.2018.2872500.
  7. Miroshnikova N. E., Petruchin G. S., Sherbakov A. V. Problems of Underwater Optical Links Modeling, 2019 Systems of Signal Synchronization, Generating and Processing in Telecommunications (SYNCHROINFO), Russia, 2019, pp. 1—9. DOI: 10.1109/SYN-CHROINFO.2019.8813996.
  8. Wu T. C., Chi Y. C., Wang H. Y., Tsai C. T., Lin G. R. Blue laser diode enables underwater communication at 12.4 Gbps, Scien­tific reports, 2017, vol. 7, article 40480. DOI: 10.1038/srep40480.
  9. Pontbriand C., Farr N., Hansen J. et al. Wireless data harvesting using the AUV Sentry and WHOI optical modem, Proceeding of the OCEANS 2015 — MTS/IEEE Washington, Washington, DC, USA, 2015, pp. 1—6. DOI: 10.23919/OCEANS.2015.7401985.
  10. Khan S., Rahmani H., Shah S. A. A. et al. A guide to convolutional neural networks for computer vision, Synthesis Lectures on Computer Vision, 2018, vol. 8, no. 1, pp. 1—207.
  11. Bulow H., Birk A. Fast and robust photomapping with an Unmanned Aerial Vehicle (UAV), Proceeding of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA, 2009, pp. 3368—3373. DOI: 10.1109/IROS.2009.5354505.
  12. Bulow H., Birk A., Unnithan V. Online generation of an underwater photo map with improved fourier mellin based registration, Proceeding of the OCEANS 2009-EUROPE, Bremen, Germany, 2009, pp. 1—6. DOI: 10.1109/OCEANSE.2009.5278193.
  13. Ait-Aoudia S., Mahiou R., Djebli H., Guerrout E. Satellite and aerial image mosaicking — A comparative insight, Proceeding of the 2012 16th International Conference on Information Visualisation, Montpellier, France, 2012, pp. 652—657. DOI: 10.1109/IV.2012.113.
  14. Fischler M. A., Bolles R. C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM, 1981, vol. 24, iss. 6, pp. 381—395. DOI: 10.1145/358669. 358692.
  15. Lowe D. G. Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, 2004, vol. 60, iss. 2, pp. 91—110. DOI: 10.1023/B:VISI.0000029664.99615.94.
  16. Pathak K., Pfingsthorn M., Btlow H., Birk A. Robust estimation of camera-tilt for iFMI based underwater photo-mapping using a calibrated monocular camera, Proceeding of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 2013, pp. 5111—5116. DOI: 10.1109/ICRA.2013.6631307.
  17. Gracias X., Negahdaripour S. Underwater mosaic creation using video sequences from different altitudes, Proceedings of the OCEANS 2005 MTS/IEEE, Washington, DC, USA, 2005, pp. 1295—1300. DOI: 10.1109/OCEANS.2005.1639933.
  18. Mizuno K., Tabeta S., MatsumotoY. et al. Development of a towed optical camera array system (SSS: Speedy Sea Scanner) for sea environmental monitoring, Proceedings of the 2018 OCEANS — MTS/IEEE Kobe Techno-Oceans (OTO), Kobe, Japan, 2018, pp. 1—5. DOI: 10.1109/OCEANSKOBE.2018.8558885.
  19. Kamaev A. N., Smagin S. I. Image stitching in the creation of photographic maps of the seabed, Vichislitel'nie Tehnologii, 2018, vol. 23, no. 1, pp. 61—72 (in Russian).
  20. Kamaev A. N. Automatic stitching of images obtained from the AUUV during the exploration of the seabed, Tehnicheskie Prob-lemi Osvoeniya Mirovogo Okeana, 2013, vol. 5, pp. 474—479 (in Russian).
  21. Filaretov V., Yukhimets D. Synthesis method of control system for spatial motion of autonomous underwater vehicle, International Journal of Industrial Engineering and Management, 2012, vol. 3, pp. 133—141.
  22. Osokin V. V., Gorojanin N. E., Vavilov V. R., Kamilov D. U. Fundamentals of the implementation of online geographical maps, Intellektual'nie Systemi. Teoriya i prilojeniya, 2015, vol. 19, no. 2. pp. 97—122 (in Russian).
  23. Egorov P. V. Description of a method for building a library for displaying a raster map that is invariant to the formats of the target geospatial data, Trudi Nauchno-issledovatel'skogo Instituta Systemnih Issledovanii Rossiiskoi Akademii Nayk, 2021, vol. 11, no. 4, pp. 38—47. DOI: 10.25682/NIISI.2021.4.0004 (in Russian).
  24. Timohin P. Yu., Mikhailyuk M. V. Ultra-large textures for highly realistic visualization of virtual landscapes, Infornatsionnie Technologii i Vichslitelnie Systemi, 2013, no. 3, pp. 46—54.
  25. Leaflet: Overview: Leaflet — a JavaScript library for interac­tive maps, available at: https://leafletjs.com/
  26. Reis D., Kupec J., Hong J., Daoudi A. Real-time flying object detection with YOLOv8, arXiv preprint arXiv:2305.09972, 2023. DOI: 10.48550/arXiv.2305.09972.
  27. OpenCV: Introduction: Open-Source Computer Vision, available at: https://docs.opencv.org/3.4/d1/dfb/intro.html
  28. Krasnopevtsev B. V. Photogrammetry, Moscow, UPP "Rep-tographiya" MIIGAiK, 2008, 160 p. (in Russian).
  29. Deutsch L. P. DEFLATE Compressed Data Format Speci­fication version 1.3, IETF, 1996, pp. 1. DOI: 10.17487/RFC1951.