Run Airflow Dag For Each File

So I have this quite nice DAG in airflow which basically runs several analysis steps (implemented as airflow plugins) on binary files. A DAG is triggert by an ftp sensor which just

Solution 1:

Use 2 DAGs to separate the sensing step from analysis steps.

DAG 1:

sensor waits for new file on ftp -> once new file lands, use TriggerDagRunOperator to trigger DAG 1 itself -> use TriggerDagRunOperator to trigger DAG 2

DAG 2:

do the analysis steps for the file