Developing Event-Driven Microservices in Java, JavaScript, Python, .NET, and Go with AQ/TEQ in the Oracle Database

by | Oct 11, 2021 | Java and Python Development, Software Development | 0 comments

This is the second in a series of blogs on data-driven microservices design mechanisms and transaction patterns with the Oracle converged database. The first blog illustrated how to connect to an Oracle database in Java, JavaScript, Python, .NET, and Go as succinctly as possible. The goal of this second blog is to use that connection to receive and send messages with Oracle AQ (Advanced Queueing) queues and topics and conduct an update and read from the database using all of these same languages.

Advanced Queuing (AQ) is a messaging system that is part of every Oracle database edition and was first released in 2002. AQ sharded queues introduced partitioning in release 12c and is now called Transaction Event Queues (TEQ).

Microservices increasingly use event-driven architectures for communication and related to this many data-driven systems are also employing an event sourcing pattern of one form or another. This is when data changes (eg a SQL command to “insert order”) are sent via events that describe the data change (eg an “orderPlaced” event) that are received by interested services. Thus the data is sourced from the events and event sourcing in general moves the source of truth for data to the event broker. This fits nicely with the decoupling paradigm of microservices.

It is very important to notice that there are actually two operations involved in event sourcing, the data change being made and the communication/event of that data change. There is, therefore, a transactional consideration and any inconsistency or failure causing a lack of atomicity between these two operations must be accounted for. This is an area where TEQ has an extremely significant and unique advantage as it, the messaging/eventing system, is actually part of the database system itself and therefore can conduct both of these operations in the same local transaction and provide this atomicity guarantee.

More details on AQ can be found at the blog Apache Kafka versus Oracle Transactional Event Queues (TEQ) as Microservices Event Mesh.

The examples provided in this blog will succinctly show the pattern just described using an order/inventory scenario:

A local transaction is started.

A message with a JSON payload for an order is dequeued from the orderqueue queue.
The inventory for the item requested in the order is reduced in the inventory table and the inventory location is returned.
A message with a JSON payload (including the inventory status and location) is enqueued on the inventoryqueue queue.
The local transaction is committed.
Note that the SYS.AQ$_JMS_TEXT_MESSAGE type is interoperable between all languages and supports a JSON payload making it perfect for writing microservices that work together regardless of language.