×
Data Processing

From ETL to ELT: Modernising Pipelines for High-Volume Metrics

This article explores how the ELT (Extract, Load, Transform) approach modernizes data pipelines, offering greater scalability, flexibility, and speed for today's demanding analytics workloads.

MaxCompute Streaming Insert: Practices and Breakthroughs in Migrating High-volume Data Stream Ingestion

This article presents the architecture, optimizations, and business impact of MaxCompute Streaming Insert in migrating high-throughput real-time data streams from BigQuery.

Best Practice of Cross-border Data Warehouse Migration: MaxCompute-based Multi-tenant Big Data Platform Architecture

This article introduces how to build a MaxCompute-based multi-tenant big data platform on Alibaba Cloud with cross-tenant data access and control.

Smooth Conversion and Migration of 100,000 SQL Statements Based on Enhanced MaxCompute SQL Syntax and Functions

This article introduces MaxCompute’s enhanced SQL and BigQuery-compatible features for smoothly converting and migrating 100,000 SQL statements from BigQuery.

Best Practice of Cross-border Data Warehouse Migration: MaxCompute's Nearline Query Solution Boosts Query Efficiency for Real-time Scenarios

This article introduces how MaxCompute's MaxQA solution significantly boosts query efficiency for real-time scenarios by optimizing architecture and leveraging dedicated resources.

Real-Time Lakehouse Solutions: Apache Flink & Apache Paimon Integration

Alibaba Cloud presents key optimizations in Flink-Paimon real-time lakehouse architecture, including the Variant data type for efficient semi-structur...

Best Practice of Cross-border Data Warehouse Migration: Enterprise-grade Upgrade to MaxCompute: Enhanced Cross-domain Access Control and Data Security

This article introduces MaxCompute’s upgraded enterprise permission system with cross-domain access control, hierarchical inheritance, and policy‑tag-based dynamic data masking.

MaxCompute SQL Execution Engine's Comprehensive Refactoring of Complex Data Type Processing Ensures Smooth Migration from BigQuery

This article introduces MaxCompute's comprehensive refactoring of its SQL execution engine to optimize complex data type processing, enabling smooth and high-performance migration from BigQuery.

Alibaba Cloud, Ververica, Confluent, and LinkedIn Join Forces on Streaming Innovation for Agentic AI

Apache Flink Agents: A landmark collaboration to build a scalable, production-grade framework for event-driven streaming agents powered by Apache Flink.

Best Practices for Ray on ACK: Secure Deployment of AI Data Processing/Training/Inference Environments

This article provides best practices for securely deploying and operating Ray on Alibaba Cloud ACK for AI data processing, training, and inference environments.

Right Approach to JSON Log Analysis: A Hands-on Guide to Efficient Practices with Alibaba Cloud SLS

This article describes the best practices for processing and analyzing JSON logs in Alibaba Cloud Simple Log Service (SLS).

Evolution of Processing: SPL One-Click Acceleration for Log-to-Metric Conversion

This article introduces SPL's one-click acceleration for converting logs to metrics, enhancing performance, observability, and data processing efficiency.

Building a Real-Time Advertising Lakehouse: Alibaba Mama's Practice with Flink & Paimon

We will first introduce the business background of Alibaba Mama's advertising platform, then explore the design and evolution of its real-time advertising system and data lake architecture.

Fluss: Redefining Streaming Storage for Real-time Data Analytics and AI

Explore Apache Fluss, the revolutionary streaming storage solution bridging traditional systems and lakehouse architectures for real-time data analytics and AI.

FLIP-9: Trigger Language - Apache Flink Rule Definition Guide

description" content="Learn about FLIP-9 proposal for Apache Flink trigger language. Discover why this rule language for Flink triggers was shelved an.

Apache Flink Broadcast Variable Optimization: FLIP-5's Approach to Reducing Network Overhead

Learn Apache Flink FLIP-5 broadcast variable optimization strategies for reducing network overhead. Discover performance lessons and modern scaling approaches for production stream processing.

Apache Flink FLIP-4: Enhanced Window Evictor for Flexible Data Eviction Before/After Processing

Master Apache Flink FLIP-4 enhanced window evictor for flexible data eviction. Learn real-time quality control, anomaly detection, and production window processing strategies.

Mastering Flink State Scaling: FLIP-8 Non-Partitioned State Management for Distributed Systems

Master Apache Flink FLIP-8 scalable non-partitioned state management. Learn dynamic scaling solutions, OperatorStateStore implementation, and state re...

Starting from o11y 2.0: The "More, Faster, Better, Cheaper" Approach to Big Data Pipelines

This article explains how Alibaba Cloud's Simple Log Service (SLS) provides a "more, faster, better, and cheaper" approach to big data pipelines to meet the demands of modern Observability 2.

Unlocking Data Value Without Compromise: Privacy-Enhancing Computation on Alibaba Cloud

The article introduces how Alibaba Cloud employs PEC to securely process data while maintaining privacy compliance and enhancing business collaboration.