Apache Kafka gets SQL support

Apache Kafka, the key component in many data pipeline architectures, is getting SQL support. San Francisco-based Confluent has released an open source streaming SQL engine called KSQL that enables developers with continuous, interactive queries on Kafka.

The latest announcement is quite important for businesses that need to respond to SQL queries on Apache Kafka. The same functionality was earlier limited to Java or Python APIs.

KSQL provides an easier way to leverage the real-time data on Kafka. Any developer who is familiar with SQL can readily use KSQL on Kafka to build solutions. The platform has a familiar syntax structure and does not require mastery of any complex infrastructure or a programming language. Moreover, KSQL coupled with Kafka’s scalable and reliable environment is expected to add a lot of value to Kafka users.

“Until now, stream processing has required complex infrastructure, sophisticated developers and a serious investment. With KSQL, stream processing on Apache Kafka is available through a familiar SQL-like interface, rather than only to developers who are familiar with Java or Python. It is the first completely interactive, distributed streaming SQL engine for Apache Kafka,” said Neha Narkhede, co-founder and CTO, Confluent, in a statement.

KSQL for streaming data is quite different from traditional rational SQL databases. The data is unbound, whereas the queries are continuously running and producing results. Confluent believes that it is easier to learn additional concepts and constructs while using a familiar language and tools.

Confluent has made major progress with Kafka. The platform has become the top choice for real-time enterprise application development. It has also become more than just data ingestion in recent years.