Senior Software Engineer | Data Processing

Job description
The Trade Desk is a global technology company with a mission to create a better, more open internet for everyone through principled, intelligent advertising. Handling over 1 trillion queries per day, our platform operates at an unprecedented scale. We have also built something even stronger and more valuable: an award-winning culture based on trust, ownership, empathy, and collaboration. We value the unique experiences and perspectives that each person brings to The Trade Desk, and we are committed to fostering inclusive spaces where everyone can bring their authentic selves to work every day.

Do you have a passion for solving hard problems at scale? Are you eager to join a dynamic, globally- connected team where your contributions will make a meaningful difference in building a better media ecosystem? Come and see why Fortune magazine consistently ranks The Trade Desk among the best small- to medium-sized workplaces globally.

What We Do

Our Software Engineers are end-to-end owners who will participate in all aspects of designing, building, and delivering data-focused products for our stakeholders. 

In TTD we maintain and manage platform for Big Data processing combining together open-source solutions of Airflow, Spark and K8s, in-house build functionality to improve data handling, orchestration and security and cloud-based compute spanning three major cloud providers (AWS, Azure, AliCloud). We are serving massive internal community as a source of support and as mediators between their needs and technical capabilities of solutions that we build and use. We are responsible for most of SDLC components and parts in that area starting from documentation and best practices to metrics and alerts. Our systems perform every day, 24/7, across the globe.

What you’ll do:

Some of the work that you will be doing to help us deliver on our mission is:
• Design, build, deliver, and own full lifecycle development software from inception to production in all areas of data processing.
• Build components and processes for BigData processing that are regularly used by others.
• Maintain, improve and evolve Big Data processing platform composed of Airflow (Python), Spark (Scala), PySpark, in-house solution for data processing (C# on .NET 6).
• Support internal community of data engineers on their endeavor of use of the platform in most efficient and effective way.
• Contribute to instilling into engineers and applying best practices to optimize Spark jobs.
• Build generalised ETL pipelines to optimize internal workflow.
• Participate in and lead projects with a high level of technical ambiguity.
• Lead others through example, mentorship, teaching.
• Participate in hiring activities

We are a global team with different backgrounds, experiences and perspectives. To complement this team, you will welcome ideas that are different from your own and be well-versed in building from common ground to value, seek out, and foster invisible and visible dimensions of diversity.

Who you are:

Our culture is much deeper than just having fun together (though, we do that well too...). We take pride in our engineers being trust-builders, generous givers, scrappy problem solvers, and gritty pursuers of excellence. You do not have to meet all the requirements below, but we believe that people who meet most of them will have a higher likelihood of succeeding in this role:
• * You have proven experience in data engineering with high-volume, heterogeneous data, ideally within distributed systems such as AWS or Azure.
• You are familiar with data processing frameworks like Spark
• You have strong programming skills in Java or Scala.
• You are committed to agile software processes, data-driven development, reliability, and responsible experimentation.
• You have hands-on experience with orchestration tools like Airflow or similar
• You have excellent problem-solving and communication skills.

Variety of technical opportunity is one of the best things about working at The Trade Desk as a software engineer which is why we do not expect you to know every technology we use when you start. What we care about is that you can learn quickly and find solutions to complex problems using the optimum tools for the job. What you know is less important than how well you learn and innovate. We don't need engineers who know all the answers; we need engineers who can invent the answers no one has thought of yet, to the questions yet to be asked.