Staff Data Engineer - Craft Education

If you're passionate about building a better future for individuals, communities, and our country-and you're committed to working hard to play your part in building that future-consider Craft Education as the next step in your career.

Craft Education is on a mission to solve skilled labor shortages by powering work-based learning pathways for all.

At Craft, we're revolutionizing the apprenticeship degree, combining on-the-job learning with accredited instruction to create innovative educational pathways that accommodate working professionals and meet employer needs. Our flagship product - Craft Connect - helps organizations administer apprentice degree programs and address mission-critical data and reporting needs. Through the same platform, Craft is also transforming how on-the-job learning converts into academic credits.

Our team of technology, education and workforce professionals also provides technical assistance to organizations looking to launch and manage apprenticeship degree programs. We are working tirelessly to accelerate the expansion of these programs along with the data infrastructure that underpins them.

If you're looking to join the work-based learning revolution, we'd love to talk with you. At Craft, you'll have the opportunity to solve hard problems in a high-growth startup environment and make a lasting impact on the future of education and workforce development. We couldn't be more excited to advance this work as a team of innovative, collaborative and mission-oriented professionals - we hope you'll consider joining us.

The salary range for this position takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs.

At WGU, it is not typical for an individual to be hired at or near the top of the range for their position, and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is:

Job Description

The Staff Data Engineer for Craft Education will lead a team of Data Engineers to produce system architecture, data ingest/egress pipelines in support of customer and business intelligence analytics. You should be agnostic to tools and should be able to supervise, design, architect and code using tools like Apache Kafka, Airflow, or Spark and other cloud technologies.

The core responsibility for this position includes supervision of data engineering technical aspects, design of data and system architecture for the Data Lake and data warehouse, supervision of the technical aspects of a data engineering team and projects encompassing dimensional and normalized data modeling.

The Staff Data Engineer will improve technical standards in the environment ensuring optimal use of data warehouse and other data stores to solve business problems. They will serve as the lead engineer and go to person for all aspects of the data engineer team including solution architecture of data systems.

Essential Functions and Responsibilities:

• Supervise work on cloud technologies and architect scalable and performant Data Warehouse/Lake systems
• Establish design and methodology for database build processes.
• Supervise the architecture and design of complete data model solutions.
• Supervise necessary data protection and security processes.
• Create and design extract processes for data access layer.
• Translate business problems/information requirements accurately to logical/physical data models aligning with customers' data architecture standards.
• Supervise and perform research and analysis to find solutions for complex business problems.
• Monitor job performance and fine tune Spark SQL queries as appropriate on a regular basis.
• Supervise the profiling of data, the publishing of data profiles and corrective actions if required to ensure data quality.
• Supervise and perform documentation / reverse engineering / analysis of data mapping using data integration code/tools.
• Work with APIs for data wrangling and integrations with other systems data in the EDW.
• Perform impact analysis using Data Integration/Data Virtualization tool repositories, DB data dictionary, UNIX scripts and frontend code on versioning systems.
• Analyze / research data on multiple platforms as wells as multiple heterogeneous databases including custom developed databases.
• Positively impact projects by completing tasks assigned on time.
• Communicate technical and domain knowledge as it relates to work, to both technical and non-technical audiences.
• Ingest and transform structured, semi-structured, and unstructured data from sources including relational databases, NoSQL, external APIs, JSON, XML, delimited files, and more.
• Support business and functional requirements and translate these requirements into robust, scalable, solutions.
• Collaborate with engineers to help adopt best practices in data system creation, data integrity, test design, analysis, validation, and documentation.
• Help continually improve ongoing reporting and analysis processes, automate, or simplify self-service modelling and production support for customers.
• Performs other related duties as assigned.

Knowledge, Skill and Abilities:

• Solid design skills for creating non-competing processes for data transfer between OLTP and OLAP sources/sinks
• Expertise with analytical reporting tools, preferably Metabase
• Mastery in code based ETL/ ELT tools for importing and exporting data across disparate systems.
• Expertise in analytic skills related to working with unstructured datasets.
• Use of industry best practices for code development, testing, implementation and documentation.
• Ability to evaluate and prioritize work based on the organization's needs.
• Ability to supervise cross team projects to accomplish data integrations and pipelines.
• Supervisory abilities for data engineering team with respect to technical design and architecture.
• Excellent verbal & written communication, along with technical documentation
• Ability to work and deliver in a team environment
• Ability to manage the use of tools like Jira, Confluence, GitHub
• Architect and Develop processes for audit of Data Integrity
• Ability to mentor Associate/Senior/Data Engineer in data pipeline architecture and coding standards
• Supervise Validation and testing to analyze and debug issues
• Mastery of AWS cloud technologies and REST APIs
• Mastery of relational SQL and NoSQL databases
• Mastery with object-oriented/object function scripting languages: Python, Java, Scala
• Mastery of big data tools: Hadoop, Spark, Kafka, Databricks, etc.

Organizational or Student Impact:

• Recommends and implements changes in technical/business processes; identifies areas for improvement.
• Helps lead/coordinate extremely complex technical projects and programs and leads development and implementation of innovative solutions for specialized technical issues.
• Works proactively; identifies and helps prevent/ solve problems that may cross disciplines.
• Fully understands and quantifies project risks with impact. Identifies, generates, and implements innovative solutions.

Problem Solving & Decision Making:

• This individual accomplishes goals and objectives independently.
• Builds and leads teams, influencing decisions and results.
• Understands and aligns to Agile practices, knowing that Agile is a team level framework that is not rigid, but flexible to support the notion of continuous delivery via iterative planning, design, implementation and operational delivery with quality.
• Uses discretion to fully scope, design, and implement solutions to complex technical problems.
• The individual provides regular technical advice and direction to technical teams and management.
• Models and helps set high standards for effective interactions with internal and external individuals.

Communication & Influence:

• Communicates with parties within and outside of their job function and typically has responsibilities for communicating with parties external to the organization.
• Works to influence others to accept and understand new concepts, practices, and approaches. Requires ability to communicate with executive leadership regarding matters of significant importance to the organization.
• This individual may conduct briefings with senior leaders within the technical function.

Leadership:

• Frequently responsible for providing guidance, coaching, and training to other employees across the Company within the area of expertise.
• Responsible for managing large, complex project initiatives or strategically important solutions to the organization, involving large cross-functional teams.
• Understands and works, with their team, to demonstrate intention and delivery via metrics. DORA metrics are cultural in nature and we use them to drive high performance teams, empowered by their own ideas on how to adjust to improve measures.
• May have direct reports but generally fewer than three.

Job Qualifications:

Minimum Qualifications:

• M. S. in Business, Management Information Systems, Computer Science, or a related field, or an equivalent combination of experience and training.
• Six or more years of experience as a Data Engineer, Data Integration, Big Data, or Business Intelligence, Software Engineer
• Prior experience acting as a Team Lead, where you run the team from a technical perspective, creating ownership and accountability for high performance delivery. You will be a partner with your manager and will be the primary point of contact for team activities/deliveries
• The individual is acknowledged within the group as a subject matter expert.
• Typically requires a University Degree or equivalent experience.

Preferred Qualifications:

• Strong experience with early stage companies distance education and distance learning students is preferred.
• 6 years of prior relevant experience.

Physical Requirements:

• Prolonged periods sitting at a desk and working on a computer.
• Must be able to lift up to 15 pounds at times.

#LI-ONSITE

Position & Application Details

How to Apply: If interested, an application will need to be submitted online. Internal WGU employees will need to apply through the internal job board in Workday.

Additional Information

Disclaimer: The job posting highlights the most critical responsibilities and requirements of the job. It's not all-inclusive.

Accommodations: Applicants with disabilities who require assistance or accommodation during the application or interview process should contact our Talent Acquisition team at recruiting@wgu.edu.

Equal Employment Opportunity: All qualified applicants will receive consideration for employment without regard to any protected characteristic as required by law.