A system for efficient time series creation from textual data

April 17, 2019
Founded by:
Narodowe Centrum Badań i Rozwoju, Tango 3 (ID: 416322)
Duration:
12 months
Leader:
dr inż. Krzysztof Kaczmarski (Warsaw University of Technology)
Team:
inż. Artur Niewiadomski (Warsaw University of Technology) inż. Stanisław Piotrowski (Warsaw University of Technology) inż. Krystian Rytel (Warsaw University of Technology) Investin Sp z o.o.

The aim of the project is to carry out conceptual and R&D works investigating the possibilities of implementing a textual logs processing system using GPU processors. Such logs can collect billions of entries, which should be translated into numeric information and stored in a time series database in a limited time (possibly real-time). Distributed systems based on the map-reduce method are commonly used for this purpose, in order to obtain the necessary efficiency. Utilization of algorithms that arose as a result of the base project will allow to build a prototype on only one machine equipped with a computational device. Such a solution will lead to obvious savings in electricity consumption, space, resources and personnel costs. The new solution will be tested on data from an industrial partner. The analyzes necessary to establish cooperation with the partner regarding the system performance, implementation costs and benefits from the application will be made.

Plakat