Job description

About Us
LightBox is on a mission to modernize the real estate industry. We provide commercial, geographical, spatial, and environmental building data on a single platform. Our definitive data and intuitive products are transforming the way organizations of all sizes solve problems.

Position Overview
We are currently looking for an innovative Data Engineer to join our LightBox data integration team. LightBox ingests thousands of datasets and processes billions of data elements from petascale amounts of data, served out through tens of billions of API transactions yearly. You will be working on a highly motivated team that is responsible for data modelling and data architecting to solve various use case scenarios. Including exciting work with highly scalable data ingestion pipelines while working in a team with extremely qualified and accomplished data and software engineers to build, enhance, and maintain our data platform that supports our best in class products.

What you will do and achieve
The role of Data Engineer includes, but are not limited to:

• Model and architect data to solve business problems.
• Analyze and resolve technical and application problems.
• Create data pipelines to prepare and maintain data to solve specific use cases.
• Adhere to high-quality development principles while delivering solutions on-time and on-budget.
• Migrate existing workflows to newer infrastructures.
• Analyze use cases and propose solutions to meet business objectives.

Who you are
Education

• Bachelor's degree or better in Computer Science or equivalent.
• Must have an excellent academic record with a good grounding in Software Engineering theory including at least one modern programming language.
• Familiar with Standards, concepts, practices, and procedures within the field of Computer Science.

Experience

• 3 - 5 yrs. experience as a data engineer.

Key Knowledge & Skills

• Strong RDMS skills, utilizing MS-SQL, Postgres or Snowflake.
• Proven experience designing databases, troubleshooting performance issues, and writing advanced SQL queries and stored procedures for efficient data extraction and manipulation.
• Experience building scalable and maintainable data intensive applications.
• At least one modern programming language, preferably Python.
• Pipeline orchestration technology, such as Prefect, Luigi or Airflow.
• Pipeline transformation tools, using Python and Pandas library.
• Big data technologies, such Databricks, Google BigQuery, Apache Spark, or Amazon EMR.

Core Competencies

• Keen interest in data engineering and a “tinkering” mindset.
• Driven to continually learn about and incorporate new technologies.
• Thrive in a self-driven environment.
• Understanding and integrating human and machine workflows.
• Data Lake & Warehouse Modeling.
• Git, Docker, Kubernetes.
• Cloud infrastructure, such as Azure, AWS, or Google Cloud.
• Development on Linux.

Other Desirable Attributes

• Location - Spatial Data.
• Distributed systems - storage, compute & access patterns.
• Unstructured Data extraction - pdf, scraping.

Job Type: Full-time

Pay: $80,000.00 - $120,000.00 per year

Benefits:

• 401(k)
• 401(k) matching
• Dental insurance
• Flexible schedule
• Flexible spending account
• Health insurance
• Health savings account
• Life insurance
• Paid time off
• Parental leave
• Professional development assistance
• Referral program
• Relocation assistance
• Vision insurance

Schedule:

• Monday to Friday

Supplemental pay types:

• Bonus pay

Experience:

• Big data: 1 year (Required)
• Python: 2 years (Required)
• SQL: 2 years (Required)
• Data lake: 1 year (Required)

Work Location: Remote

blackflymedia.com is the go-to platform for job seekers looking for the best job postings from around the web. With a focus on quality, the platform guarantees that all job postings are from reliable sources and are up-to-date. It also offers a variety of tools to help users find the perfect job for them, such as searching by location and filtering by industry. Furthermore, blackflymedia.com provides helpful resources like resume tips and career advice to give job seekers an edge in their search. With its commitment to quality and user-friendliness, blackflymedia.com is the ideal place to find your next job.