Journey to Real-Time Analytics - Apache Doris: One Engine to Rule Them All

The previous article in this series ended with a question: if a table format handles data persistence and updates, what answers queries with low latency?

Trino is a common answer. But Trino does not operate alone. It is surrounded by a catalog, Object Storage, a format such as Iceberg, an ingestion layer, and a tool for organizing transformations. Each component has a clear purpose; the platform emerges from their integration.

Apache Doris takes a different approach. Instead of acting only as the Query Engine, it brings catalog, execution, storage, ingestion, and acceleration mechanisms into the same system. The relevant comparison, therefore, is not Doris versus Trino. It is Doris versus the stack Trino needs to provide serving — the layer that receives queries from BI tools and applications and returns data ready for consumption.

From an open stack to an integrated engine

Trino is a distributed Query Engine. It receives SQL, plans the query, and coordinates the work required to read and combine data. Its strength comes precisely from not requiring that data to belong to Trino: connectors make it possible to query different systems and even join them in the same query.

That flexibility covers the compute layer. The remaining responsibilities stay in other components:

This diagram is a conceptual map. It shows responsibilities, not a mandatory deployment topology.

%%{init: {
  "theme": "base",
  "themeVariables": {
    "primaryColor": "#F7F7F5",
    "primaryTextColor": "#1F2937",
    "primaryBorderColor": "#D6D3D1",
    "secondaryColor": "#EEEDE8",
    "tertiaryColor": "#FFF4DB",
    "lineColor": "#6B7280",
    "clusterBkg": "#EEEDE8",
    "clusterBorder": "#D6D3D1",
    "fontFamily": "Inter, Segoe UI, Helvetica Neue, Arial, sans-serif",
    "fontSize": "15px"
  },
  "flowchart": {"curve": "basis", "nodeSpacing": 24, "rankSpacing": 32, "padding": 8}
}}%%
flowchart TB
    ING["Ingestion (SeaTunnel/Airbyte)"]
    DATA["Object Storage (S3/GCS)<br/>+ Lakehouse format (Iceberg)"]
    CAT["Catalog (Hive Metastore)"]
    ELT["ELT (dbt)"]
    TRINO["Trino"]
    BI1["BI / internal analytics"]
    SERVING["Application serving database<br/>(PostgreSQL / MySQL)"]
    APP["Customer-facing<br/>application"]

    ING --> DATA
    DATA --> TRINO
    CAT --> TRINO
    ELT --> TRINO
    TRINO --> BI1
    TRINO --> SERVING
    SERVING --> APP

    classDef muted fill:#F7F7F5,stroke:#D6D3D1,color:#4B5563,stroke-width:1px;
    classDef active fill:#FFF4DB,stroke:#B45309,color:#1F2937,stroke-width:2px;
    class ING,DATA,CAT,ELT,BI1,SERVING,APP muted;
    class TRINO active;

• Ingestion: The first step is moving data from operational systems, events, or files into the analytical layer. This work can be continuous or batch-based, but it is not Trino’s responsibility. A pipeline must write the data to a destination Trino can query.

• Storage: Object Storage such as S3 or MinIO keeps files durable and inexpensive. It understands objects and paths, but it cannot determine on its own that a set of files represents a table, which schema is current, or which files form the latest version of the data.

• Lakehouse: Apache Iceberg organizes those files as a table. It records snapshots, schema evolution, partitions, and which files belong to each version. Different engines can then read the same data with a consistent interpretation.

• Catalog: Before reading an Iceberg table, Trino must find it. The catalog works as a directory: it maps the name used in SQL to the metadata that describes the table’s current version. This catalog can be a Hive Metastore, a REST Catalog, or another compatible implementation.

• Trino: Once the table is located, Trino determines which files to read, distributes the work, and combines the results. It is the layer that turns SQL into processing, but it does not own the storage or the table format. This separation makes it possible to replace or add engines without moving the data.

• ELT: Raw data is rarely ready for consumption. An ELT layer, often organized with dbt, transforms source tables into business models, manages dependencies, and adds practices such as testing and documentation. dbt does not replace Trino: it sends SQL for Trino to execute. The transformation result returns to storage and once again passes through the table format and catalog.

• Application serving database: A customer-facing application cannot always run complex analytical queries on every request. A common approach is to consolidate the result in Trino and materialize it in PostgreSQL or MySQL, where the application performs simpler and more predictable reads. This database adds another copy of the data and requires a process to keep it synchronized with the analytical layer.

The cost of the stack does not come from any single component. It comes from the sum of its boundaries: integration contracts, compatible versions, authentication, networking, observability, and failure recovery across projects with different lifecycles.

How Doris consolidates the same path

Doris organizes these responsibilities as parts of the same analytical database. In internal storage mode, two roles form the core of the architecture:

• Frontend: accepts connections, maintains the catalog, analyzes SQL, and creates the query plan.
• Backend: stores data and executes the portions of the plan distributed by the Frontend.

This division still allows the system to scale, while keeping catalog, storage, and execution under the same product and operational model.

The diagram groups responsibilities. A deployment may run multiple Frontend and Backend processes.

%%{init: {
  "theme": "base",
  "themeVariables": {
    "primaryColor": "#F7F7F5",
    "primaryTextColor": "#1F2937",
    "primaryBorderColor": "#D6D3D1",
    "secondaryColor": "#EEEDE8",
    "tertiaryColor": "#FFF4DB",
    "lineColor": "#6B7280",
    "clusterBkg": "#EEEDE8",
    "clusterBorder": "#D6D3D1",
    "fontFamily": "Inter, Segoe UI, Helvetica Neue, Arial, sans-serif",
    "fontSize": "15px"
  },
  "flowchart": {"curve": "basis", "nodeSpacing": 28, "rankSpacing": 34, "padding": 8}
}}%%
flowchart LR
    ING["Ingestion"]
    BI["BI / internal analytics"]
    APP["Customer-facing<br/>application"]

    subgraph DORIS["Apache Doris · internal storage"]
        direction LR
        FRONTEND["Frontend<br/>(catalog · SQL · planning)"]
        BACKEND["Backend<br/>(distributed execution · local storage<br/>indexes · materialized views)"]

        FRONTEND <--> BACKEND
    end

    ING --> FRONTEND
    BI <--> FRONTEND
    APP <--> FRONTEND

    classDef muted fill:#F7F7F5,stroke:#D6D3D1,color:#4B5563,stroke-width:1px;
    classDef active fill:#FFF4DB,stroke:#B45309,color:#1F2937,stroke-width:2px;
    class ING,BI,APP muted;
    class FRONTEND,BACKEND active;

Ingestion writes directly to tables

Doris provides native paths for continuous loads, application-driven writes, and batch imports. This does not eliminate specialized integrations in every scenario, but it reduces the need to build an external pipeline solely to place data in the database.

With the bronze_orders table already created, continuous ingestion of JSON events published to Kafka can be configured with a single command:

CREATE ROUTINE LOAD analytics.load_bronze_orders
ON bronze_orders
COLUMNS(order_id, customer_id, status, total_amount, created_at)
PROPERTIES (
    "format" = "json",
    "jsonpaths" = "[\"$.order_id\",\"$.customer_id\",\"$.status\",\"$.total_amount\",\"$.created_at\"]"
)
FROM KAFKA (
    "kafka_broker_list" = "kafka:9092",
    "kafka_topic" = "orders",
    "property.kafka_default_offsets" = "OFFSET_END"
);

Once created, Routine Load remains active and turns messages from the topic into table writes. No external scheduler is required just to repeat the load.

Unique Key tables also support updates by key. Instead of treating every change as a set of files to be reconciled during reads, Doris can resolve multiple versions of the same key during writes.

Materialized views bring transformation closer to serving

Materialized views store the result of a transformation ahead of time, preventing each query from repeating the same work. In Doris, they are part of the optimization mechanism itself and can be selected automatically when a compatible query arrives.

The example below uses bronze_orders and bronze_order_items as input tables and creates a silver layer. The materialized view keeps only paid orders, joins orders with items, and calculates the gross value of each item:

CREATE MATERIALIZED VIEW silver_order_items
BUILD IMMEDIATE
REFRESH AUTO ON SCHEDULE EVERY 10 MINUTE
DISTRIBUTED BY HASH(order_id) BUCKETS 8
AS
SELECT
    orders.order_id,
    orders.customer_id,
    DATE(orders.created_at) AS order_date,
    items.product_id,
    items.quantity,
    items.quantity * items.unit_price AS gross_value
FROM bronze_orders AS orders
JOIN bronze_order_items AS items
    ON orders.order_id = items.order_id
WHERE orders.status = 'PAID';

BUILD IMMEDIATE creates the first version from the existing data. REFRESH AUTO ON SCHEDULE EVERY 10 MINUTE refreshes the materialized view every ten minutes. This automates the transition from bronze to silver, but the data may remain stale until the next cycle.

Bronze and silver are only layer names in this example. For Doris, bronze_orders and bronze_order_items are internal tables, while silver_order_items is a queryable materialized view.

This covers part of the role played by materialized models in an ELT layer. It does not replace the full discipline of dbt — testing, documentation, lineage, and orchestration remain relevant — but it brings transformation and serving closer together within the same system.

Catalog and planning are already part of the database

Internal tables are known directly by Doris. A separate catalog does not need to be deployed just so the Query Engine can find its own data.

For external sources, Multi-Catalog makes it possible to query them without copying everything in advance. Those external systems still exist, but Doris can also take on some of the federation work that would normally justify Trino.

Storage knows which engine will read it

Doris writes its internal tables in its own columnar format in both deployment modes. What changes is where those files live: in internal storage, segments and indexes remain on the disks of Backend processes; in decoupled mode, the same file types live in shared storage, while Backend processes keep a local cache.

In both cases, Doris controls the physical organization, indexes, and read strategies. This coupling reduces openness, but it also removes intermediate steps: the Query Engine does not need to interpret a generic format maintained for multiple engines.

The MySQL protocol reduces consumption friction

Clients and BI tools can connect through the MySQL protocol. This compatibility does not make Doris SQL identical to MySQL in every detail, but it reduces the need to introduce a dedicated protocol at the consumption edge.

Doris itself can serve customer-facing applications

When consolidated data and materialized views already live in Doris, an application can query them directly through the MySQL protocol. Under this design, the same system that receives and transforms data also answers product reads, without requiring an additional copy in PostgreSQL or MySQL solely to form the serving layer.

This is what brings Doris closer to customer-facing analytics. The project presents embedded dashboards, customer and merchant portals, and data products as native use cases, supported by low latency and high concurrency.

This does not turn Doris into a general-purpose transactional database. If an application needs operational transactions, complex referential integrity, or record-by-record writes as part of the product’s main flow, PostgreSQL or MySQL still has a distinct role. The consolidation applies to analytical serving.

The trade-off between federation and integrated serving

In federated querying, the main problem is reaching data that remains distributed across different systems. In serving, the problem becomes keeping analytical models ready for repeated, concurrent reads with predictable latency.

Trino and Doris can both participate in these scenarios. The choice is not between “right” and “wrong” architectures, but between different places to carry complexity: distributing it across specialized components or concentrating more control within a single project.

Trino + Iceberg: bring the query to the data

Trino provides a common SQL layer over heterogeneous sources without requiring every dataset to move into a single database. With Iceberg, files remain in a shareable format that different engines can access.

This autonomy favors data exploration, combination, and migration. In return, ingestion, catalog, transformations, and source availability remain distributed. If an application requires a different read profile, the result may still need to be materialized in a serving layer.

Doris: bring the critical path inside

In Doris, the data essential to serving can be ingested, transformed into materialized views, and queried by the application within the same system. The critical path stays in representations prepared to answer the product’s questions repeatedly.

Multi-Catalog still provides access to external sources. The integration advantage appears when the data that supports serving is persisted or materialized in Doris, allowing it to control storage, transformation, and reads.

The level of consolidation depends on the deployment mode.

Doris internal storage

This is the most consolidated configuration. Catalog, execution, and storage belong to Doris, and fewer processes are required to form the core of the database.

%%{init: {
  "theme": "base",
  "themeVariables": {
    "primaryColor": "#F7F7F5",
    "primaryTextColor": "#1F2937",
    "primaryBorderColor": "#D6D3D1",
    "secondaryColor": "#EEEDE8",
    "tertiaryColor": "#FFF4DB",
    "lineColor": "#6B7280",
    "clusterBkg": "#EEEDE8",
    "clusterBorder": "#D6D3D1",
    "fontFamily": "Inter, Segoe UI, Helvetica Neue, Arial, sans-serif",
    "fontSize": "15px"
  },
  "flowchart": {"curve": "linear", "nodeSpacing": 28, "rankSpacing": 32, "padding": 8}
}}%%
flowchart LR
    FRONTEND_INTERNAL["Frontend<br/>catalog · SQL · planning"]
    BACKEND_INTERNAL["Backends<br/>execution + local segments and indexes"]

    FRONTEND_INTERNAL <--> BACKEND_INTERNAL

    classDef integrated fill:#FFF4DB,stroke:#B45309,color:#1F2937,stroke-width:2px;
    class FRONTEND_INTERNAL,BACKEND_INTERNAL integrated;

This is the most coherent option when the priority is launching a serving layer with few dependencies and there is no need to scale storage separately.

Doris decoupled

Decoupled mode separates compute and storage. To do so, it adds metadata services and shared storage. It has more components than internal storage.

%%{init: {
  "theme": "base",
  "themeVariables": {
    "primaryColor": "#F7F7F5",
    "primaryTextColor": "#1F2937",
    "primaryBorderColor": "#D6D3D1",
    "secondaryColor": "#EEEDE8",
    "tertiaryColor": "#FFF4DB",
    "lineColor": "#6B7280",
    "clusterBkg": "#EEEDE8",
    "clusterBorder": "#D6D3D1",
    "fontFamily": "Inter, Segoe UI, Helvetica Neue, Arial, sans-serif",
    "fontSize": "15px"
  },
  "flowchart": {"curve": "linear", "nodeSpacing": 36, "rankSpacing": 42, "padding": 10}
}}%%
flowchart TB
    FRONTEND_DECOUPLED["Frontend<br/>catalog · SQL · planning"]
    BACKEND_DECOUPLED["Backends<br/>execution + local cache"]
    OBJ["Shared storage<br/>Doris segments + indexes"]
    MDB["Metadata database"]
    MS["Meta Service"]

    FRONTEND_DECOUPLED --> BACKEND_DECOUPLED
    MDB --> MS
    MS --> BACKEND_DECOUPLED
    BACKEND_DECOUPLED --> OBJ

    classDef integrated fill:#FFF4DB,stroke:#B45309,color:#1F2937,stroke-width:2px;
    classDef external fill:#F7F7F5,stroke:#D6D3D1,color:#4B5563,stroke-width:1px;
    class FRONTEND_DECOUPLED,BACKEND_DECOUPLED,MS integrated;
    class MDB,OBJ external;

The difference from the Trino stack is that this composition is part of Doris’s official architecture. Frontend, Backend, and Meta Service belong to the same project; shared storage and the metadata database remain external dependencies. Even so, Doris defines the topology and contracts between these components. There is no need to design the integration among an independent Query Engine, table format, and catalog to form the main path.

Decoupled mode is therefore more complex than internal storage, but it still preserves a cohesion advantage over a stack assembled from separate projects.

The difference lies at the control boundary

These capabilities do not form a rigid divide. Trino can also serve high-scale, low-latency queries when its sources and architecture are prepared for them. Doris can also federate queries across external systems. The distinction lies in the responsibility each assumes by default:

If the main challenge is providing a SQL interface over heterogeneous data while preserving independence between storage and the Query Engine, Trino + Iceberg starts from a more natural position. If the challenge is maintaining a continuous path from ingestion to transformation and analytical serving with fewer contracts between systems, Doris assumes more of those responsibilities.

Load test

The architectural comparison explains why Doris can be simpler. The remaining question is whether this consolidation preserves the ability to serve analytical queries with low latency.

Three architectures over the same data

The test used the same dataset in all three configurations:

All three received the same data and executed equivalent queries. The goal was not to simulate the full variety of a production platform, but to keep the inputs equal and observe how each architecture responded to the same work.

Three query patterns

The queries were selected to exercise different analytical patterns.

1. Revenue aggregation

The first query scans non-cancelled orders and items, associates each item with its product category, and calculates monthly volume and revenue:

SELECT
    date_trunc(o.created_at, 'month') AS month,
    p.category,
    COUNT(DISTINCT o.id) AS orders_count,
    SUM(oi.quantity) AS items_sold,
    SUM(oi.quantity * oi.unit_price) AS revenue
FROM bronze.orders AS o
JOIN bronze.order_items AS oi
    ON oi.order_id = o.id
JOIN bronze.products AS p
    ON p.id = oi.product_id
WHERE o.status <> 'cancelled'
GROUP BY 1, 2
ORDER BY month, revenue DESC;

2. Order 360 view

The second query builds an analytical view of each order. It selects the most recent address for every user and joins six tables to return customer, order, item, product, and payment data:

WITH primary_address AS (
    SELECT user_id, city, state
    FROM (
        SELECT
            user_id,
            city,
            state,
            ROW_NUMBER() OVER (
                PARTITION BY user_id
                ORDER BY created_at DESC
            ) AS rn
        FROM bronze.addresses
    ) AS ranked
    WHERE rn = 1
)
SELECT
    u.id AS user_id,
    u.name AS user_name,
    a.city,
    a.state,
    o.id AS order_id,
    o.created_at AS order_created_at,
    o.status AS order_status,
    p.sku,
    p.category,
    oi.quantity,
    oi.unit_price,
    pay.method AS payment_method,
    pay.status AS payment_status
FROM bronze.orders AS o
JOIN bronze.users AS u
    ON u.id = o.user_id
LEFT JOIN primary_address AS a
    ON a.user_id = u.id
JOIN bronze.order_items AS oi
    ON oi.order_id = o.id
JOIN bronze.products AS p
    ON p.id = oi.product_id
LEFT JOIN bronze.payments AS pay
    ON pay.order_id = o.id
WHERE o.created_at >= CURRENT_DATE - INTERVAL '30' DAY
ORDER BY o.created_at DESC, o.id, oi.id
LIMIT 500;

3. Product ranking

The third query aggregates units and revenue by product. A window function then ranks products within each category and keeps the top five:

WITH product_revenue AS (
    SELECT
        p.id AS product_id,
        p.sku,
        p.name,
        p.category,
        SUM(oi.quantity) AS units_sold,
        SUM(oi.quantity * oi.unit_price) AS revenue
    FROM bronze.order_items AS oi
    JOIN bronze.products AS p
        ON p.id = oi.product_id
    JOIN bronze.orders AS o
        ON o.id = oi.order_id
    WHERE o.status <> 'cancelled'
    GROUP BY 1, 2, 3, 4
),
ranked AS (
    SELECT
        *,
        RANK() OVER (
            PARTITION BY category
            ORDER BY revenue DESC
        ) AS category_rank
    FROM product_revenue
)
SELECT
    category,
    category_rank,
    sku,
    name,
    units_sold,
    revenue
FROM ranked
WHERE category_rank <= 5
ORDER BY category, category_rank;

Scenario 1: one query at a time

Each query was executed in isolation, with one user and 29 runs per configuration. This scenario removes contention between users and exposes the latency of each kind of work.

{
  "type": "bar",
  "title": "Median latency by query without concurrency",
  "subtitle": "1 user · 29 runs per query and configuration",
  "unit": "ms",
  "categories": ["Aggregation", "Order 360", "Ranking"],
  "series": [
    { "key": "doris-internal", "name": "Doris internal storage", "values": [59, 160, 53] },
    { "key": "doris-decoupled", "name": "Doris decoupled storage", "values": [53, 160, 53] },
    { "key": "trino", "name": "Trino + Iceberg", "values": [560, 2100, 490] }
  ],
  "note": "Exact values appear above the bars."
}

Each cell shows mean / median / p95, in milliseconds:

Based on the medians, Trino took between 9.2 and 13.1 times as long as Doris, depending on the query.

The p95 indicates the time within which 95% of runs completed, adding a view of the tail without concentrating the analysis on the single slowest call. With 29 runs per combination, it should still be treated as a snapshot of this test round, not a stable production projection.

Scenario 2: ten concurrent users

In the second scenario, ten users executed a mix of the three queries.

Each configuration received exactly 500 requests.

Under continuous requests, the gap widened:

{
  "type": "bar",
  "title": "Median latency by query under concurrent load",
  "subtitle": "10 users · 500 requests per configuration",
  "unit": "ms",
  "categories": ["Aggregation", "Order 360", "Ranking"],
  "series": [
    { "key": "doris-internal", "name": "Doris internal storage", "values": [66, 230, 64] },
    { "key": "doris-decoupled", "name": "Doris decoupled storage", "values": [64, 270, 66] },
    { "key": "trino", "name": "Trino + Iceberg", "values": [1800, 3000, 1800] }
  ],
  "note": "Exact values appear above the bars."
}

Each cell shows mean / median / p95, in milliseconds:

Based on the medians, Trino took between 11.1 and 28.1 times as long as Doris, depending on the query and storage mode.

The result of 500 requests

Combining the three queries shows how each configuration performed across all 500 requests:

To complete the same work, Trino took roughly 3.38 times as long overall and delivered approximately 29% of the throughput of internal storage. None of the three configurations recorded a failure.

What the results support

Latency and resource consumption complete the comparison. The last column uses Doris internal storage as the reference:

Doris used more CPU and completed the same 500 requests in less time. Trino used approximately 2.5 to 2.9 times more memory, but remained below one quarter of a CPU core on average.

Trino + Iceberg compared with Doris internal storage

Trino’s median was 24.66 times that of Doris internal storage. By the mean, the gap was 18.49 times. The ratio narrowed in the tail but did not reverse the result: at p99, Trino was 7.83 times higher.

The Trino + Iceberg stack used roughly 2.91 times the memory of Doris internal storage. The test did not isolate which component or stage explains each difference, but it shows that, under this workload, the openness of the stack did not come with serving performance equivalent to Doris.

Doris decoupled compared with internal storage

The gap between the two Doris modes was much smaller. Decoupled mode was 14% higher than internal storage in mean latency, 3% in the median, 13% at p95, and 28% at p99. Throughput remained close, at 98% of internal storage, while memory consumption was 18% higher.

The test does not prove that Doris will always be faster than Trino. It does not cover billions of rows, other workload types, advanced tuning, or a distributed production deployment. Its conclusion is more specific: under the conditions evaluated, both Doris modes delivered a faster serving layer with lower memory consumption, and the most consolidated configuration also produced the best overall result.

From serving to real time

The test begins after the data is already available for querying. It shows that Doris can respond quickly over materialized state, but it does not measure how long a change took to reach that state.

That distinction moves Doris closer to real time without making it, by itself, a streaming architecture. In the path presented in this article, ingestion writes new data to tables and the asynchronous materialized view publishes another representation every ten minutes. The interval is shorter than a traditional batch, but control remains cycle-based: accumulate changes, process them, and publish a new state.

A smaller batch is still a batch.

This boundary brings back the problems discussed in Lakehouse Is Not the Solution. A change may still wait for the next cycle before becoming visible; reducing the interval increases the frequency of coordination and processing; and each materialization must produce a new state from its source data. Doris changes the format, reduces the number of components, and accelerates serving, but it does not eliminate the delay introduced by a micro-batch strategy.

A 64 ms query can therefore still return a state that is almost ten minutes behind. Reducing the refresh interval to one minute shortens the wait, but does not change the model: it only runs the same cycle more frequently.

An architecture designed for real time must change the unit of processing. Instead of waiting for a batch to close before producing another version of state, every change must flow through continuous transformations, update queryable state incrementally, and continue to downstream consumers without depending on the next batch boundary.

Where each architecture fits

• Doris internal storage fits when the goal is to launch a low-latency analytical layer with few components.
• Doris decoupled fits when storage elasticity justifies additional components and some overhead.
• Trino + Iceberg remains the more coherent choice when openness, interoperability between engines, and federation matter more than consolidation.

The main criterion is not which Query Engine won a chart. It is deciding where the platform should carry its complexity: in the integrations of an open stack or inside a system that controls the complete serving path.

Primary sources

• Complete load test report (Portuguese)
• Apache Doris — What is Apache Doris
• Apache Doris — High-Concurrency Point Query
• Apache Doris — Customer-Facing Analytics
• Apache Doris — Compute-Storage Decoupled: Before Deployment
• Apache Doris — Compute-Storage Decoupled: Overview
• Apache Doris — Data Catalog Overview (Multi-Catalog)
• Apache Doris — Routine Load
• Apache Doris — Stream Load
• Apache Doris — Broker Load
• Apache Doris — Unique Key Model
• Apache Doris — Sync Materialized View
• Apache Doris — Async Materialized View Overview
• Apache Doris — Async Materialized View Best Practices
• Trino
• Trino — Iceberg connector