EDS Infrastructure

At the foundation of the LCLS ecosystem lies the Data Acquisition (DAQ) system and Data Reduction Pipeline (DRP), which capture and perform initial processing of multi-GB/s data streams from advanced detectors during experimental runs. 

This raw data, primarily stored in the eXtended Tagged Container (XTC) format and its enhanced successor XTC2, feeds into a comprehensive analytical framework centered around psana, a Python-based toolkit optimized for scientific analysis of LCLS experiments. Real-time experimental feedback is provided through the Analysis Monitoring Interface (AMI), enabling scientists to visualize data quality and characteristics during active data collection. 

For post-experimental analysis, researchers leverage specialized tools including smalldata tools for working with extracted data subsets, while complex multi-step analytical processes are orchestrated through the Workflows system. The LUTE environment provides user-friendly interfaces for interacting with these components, while the streaming and Data Management (DM) systems ensure efficient movement of data between acquisition, processing, storage, and archival stages. 

Together, these integrated components form a sophisticated data architecture that enables scientists to navigate the extreme data challenges of modern X-ray science while extracting meaningful scientific insights from LCLS experiments.