Data Processing Patterns

Data processing patterns deal with how data is received, transformed, and passed along within a system. These patterns focus on efficient and reliable manipulation of information to meet specific requirements.

Wikipedia

Map-Reduce

behavioral parallelism data processing

Map-Reduce is a programming model and an associated implementation for processing and generating large datasets. It works by splitting the dataset into independent chunks, which are then processed in parallel by “map” functions. These map functions output key-value pairs. Subsequently, “reduce” …

Read More
Pipes and Filters

behavioral data processing

The Pipes and Filters pattern is a data processing paradigm where a series of independent processing components (Filters) are connected by channels (Pipes). Each Filter performs a specific transformation on the input data and passes the result to the next Filter in the pipeline. The pattern promotes modularity, …

Read More

Map-Reduce

Pipes and Filters