Why choose Flink for real-time processing

Why choose Flink
[1] Streaming data more truly reflects our lifestyle (real-time chat);
[2] Traditional data architecture is based on limited data sets (Spark is based on micro-batch data processing);
[3] Our goal: low latency, high throughput (distributed architecture, there may be confusion in the order, for example, within 1 hour of statistics, some data may still be processed, and the arrival will be delayed. A few milliseconds, this can be circumvented through settings), accuracy of results and good

Traditional data processing architecture
At the beginning, the company mainly carried out transaction processing: CRM generated events - logical processing in Order - and finally fed back to Click. The data were all obtained through a relational database. So when the data is counted, the amount of data will be large. Encountered a bottleneck. OLTP is characterized by fast response. The disadvantage is that it is not easy to expand when the amount of data is large.

Microservice architecture: The microservice architecture disassembles the system into different independent service modules, and each module uses its own independent database. This model solves the problem of business system expansion, but it also brings new problems, that is Business transaction data is too scattered in different systems, making it difficult to centrally manage the data. For applications such as data analysis or data mining within the enterprise, it is necessary to extract data from different databases and extract the data from the database. Or the business system periodically synchronizes it to the data warehouse, and then performs data extraction, conversion, and loading (ETL) in the data warehouse to build different data marts and applications for use by the business system. However, for some applications with relatively high time requirements, such as real-time report statistics, statistical results must be displayed with very low latency. For this reason, the industry has proposed a Lambda architecture solution to process different types of data.