خرید بک لینک

Vote count: 0

I have configured airflow and created some Dags and subDags that call several operators.

My trouble is that when an operators runs and finishes the job, I'd like to receive the results back in some python structure. For instance:

from airflow import DAG
from airflow.operators import HiveOperator

def subdag_callHive(parent, child, args, step,
                         user_defined_macros, path
                        ):
        dag_subdag = DAG(
            dag_id='%s.%s' % (parent, child),
            default_args=args,
            schedule_interval="@daily",
            template_searchpath=path,
            user_defined_macros=user_defined_macros,
        )

        # some work...

        HiveOperator(
            task_id='some_id',
            hiveconf_jinja_translate=True,
            hql='select field1 from public.mytable limit 4;',
            trigger_rule='all_done',
            dag=dag_subdag,
        )

        retu dag_subdag  

The function subdag_callHive is called from another python script where the main Dag is defined and all the other parameters needed.

I just need would like to be able to get the result from the HiveOperator (*select * from public.mytable limit 4;*) that would be 4 values in this case.

Is this possible? if So, how can it be accomplished.

asked 53 secs ago

برچسب: نویسنده: استخدام کار تاريخ: يکشنبه 6 تير 1395 ساعت: 20:53

صفحه بندی