Astronomer

Data Pipelines with Apache Airflow

5 followers

Data Pipelines with Apache Airflow

5 followers

Building a Data Engineering platform to capture, transform, and route your data. Built Airflow, Kubernetes, Kafka, and other open-source components.

This is the 4th launch from Astronomer. View more

Apache Airflow on Astronomer: Enterprise Edition

The easiest way to run Apache Airflow on your infrastructure

Our Enterprise Apache Airflow includes one-click DAG deployment through our Astro CLI, a full monitoring stack, access to our library of hooks and operators, and professional support. The entire package is Kubernetes-deployable; it can be hosted entirely on your infrastructure.

Apache Airflow on Astronomer: Enterprise Edition gallery image

Launch tags:

Open Source•Developer Tools•Tech

Launch Team

Dylan Kim

I saw Google launched Cloud Composer yesterday, which looks like their version of managed Airflow. What does this have that differentiates it from that?

Report

7yr ago

Ry Walker

Tembo Cloud

Maker

@dylan_kim Hi Dylan - we love that Apache Airflow is getting love from Google, and we're excited to collaborate with Google Composer team to push Airflow forward, along with the excellent core Apache Airflow team. 1) The biggest difference now between Google Composer and Astronomer Airflow is that you have access to all the code used in our Enterprise Edition, so it's fully-hackable. 2) Astronomer EE runs anywhere that Kubernetes runs, which means that as AWS/Azure/Digital Ocean rolls out their Kubernetes services, you can run Airflow there just as easily as you can on Google Cloud. All that being said, Google Cloud is a great place to run Kubernetes applications, so the first guide we wrote was how to get Astronomer EE running on GCP (http://enterprise.astronomer.io/...).

Report

7yr ago

Ry Walker

Tembo Cloud

Maker

I’m Ry Walker, CEO. We built this version of Apache Airflow on Astronomer as a self-hosted, next-generation version of our managed Airflow. Because one-size-fits-all solutions from SaaS tools often don’t provide enough flexibility, this Kubernetes-deployable version of the platform will help savvy data teams gain full control of their data orchestration stack without having to pay the full price of going at it completely alone. With our product, you get the complete package- Airflow, Celery, Flower, Grafana and Prometheus - deployable to any Kubernetes cluster. With this version, we've added the ability to create multiple Airflow deployments from the CLI, support for custom airflow images, and a host of other goodies. And as always, you get the source code to all of it to modify however you want. Feel free to email me (ry@astronomer.io) if you'd like to kick tires.

Report

7yr ago