Skip to content Skip to sidebar Skip to footer
Showing posts with the label Airflow

Not Able To Setup Airflow, Getting Error While "initiating Airflow Database"

Not able to setup airflow, getting error while 'Initiating Airflow Database' . I am getting… Read more Not Able To Setup Airflow, Getting Error While "initiating Airflow Database"

Airflow Cron Expression Is Not Scheduling Dag Properly

I am exploring Airflow to be used as a cron so that I could use its other features while setting up… Read more Airflow Cron Expression Is Not Scheduling Dag Properly

Providing Context In Triggerdagrunoperator

I have a dag that has been triggered by another dag. I have passed through to this dag some configu… Read more Providing Context In Triggerdagrunoperator

Run Airflow Dag For Each File

So I have this quite nice DAG in airflow which basically runs several analysis steps (implemented a… Read more Run Airflow Dag For Each File

Templatesyntaxerror: Unexpected Char U'\\' At 205 Airflow Macros

I am getting the following error : File ' ', line 1, in template TemplateSyntaxError: unex… Read more Templatesyntaxerror: Unexpected Char U'\\' At 205 Airflow Macros

Airflow Bashoperator Can't Find Bash

I'm using Airflow in Centos 7, using Python 3.7. When I run a Bash command through BashOperator… Read more Airflow Bashoperator Can't Find Bash

How >> Operator Defines Task Dependencies In Airflow?

I was going through Apache Airflow tutorial https://github.com/hgrif/airflow-tutorial and encounter… Read more How >> Operator Defines Task Dependencies In Airflow?

Etl Model With Dags And Tasks

I'm trying to model my ETL jobs with Airflow. All jobs have kind of the same structure: Extrac… Read more Etl Model With Dags And Tasks

Python - Attributeerror: 'nonetype' Object Has No Attribute 'execute'

I am trying to run a python script that logs into Amazon Redshift DB and then execute a SQL command… Read more Python - Attributeerror: 'nonetype' Object Has No Attribute 'execute'

How To Run Airflow With Celeryexecutor On A Custom Docker Image

I am adding airflow to a web application that manually adds a directory containing business logic t… Read more How To Run Airflow With Celeryexecutor On A Custom Docker Image

Airflow: How To Use Trigger Parameters In Functions

We are using Airflow's KubernetesPodOperator for our data pipelines. What we would like to add … Read more Airflow: How To Use Trigger Parameters In Functions

Store And Access Password Using Apache Airflow

We are using Airflow as a scheduler. I want to invoke a simple bash operator in a DAG. The bash scr… Read more Store And Access Password Using Apache Airflow

"Handling Signal: Ttou" Message While Running DAG In Airflow

I have created sample DAG, where I had DAG config as below. default_args = { 'owner': &… Read more "Handling Signal: Ttou" Message While Running DAG In Airflow

Paperboy Throwing Error: TypeError: The JSON Object Must Be Str, Not 'bytes'?

I have installed paperboy using steps given at https://github.com/timkpaine/paperboy.git and execut… Read more Paperboy Throwing Error: TypeError: The JSON Object Must Be Str, Not 'bytes'?

Retrieve Full Connection URI From Airflow Postgres Hook

Is there a neater way to get the complete URI from a Postgres hook? .get_uri() doesn't include … Read more Retrieve Full Connection URI From Airflow Postgres Hook

Store And Access Password Using Apache Airflow

We are using Airflow as a scheduler. I want to invoke a simple bash operator in a DAG. The bash scr… Read more Store And Access Password Using Apache Airflow