Job Details

About the job: Data Engineer (Python, Java) Remote (50%) Columbia SC

Authorized to work in the US

No H1B

Government Experience: Preferred

Overall Exp 11+ yrs

The South Carolina Department of Health & Human Services (SCDHHS) is the State Medicaid Agency for South Carolina. This agency is focused on modernizing the States Medicaid Management Information System (MMIS). The modernization involves significant transformation of culture, processes, and technology, supporting a shift from fee-for-service claims to health outcomes through managed care programs.

Scope of the project:

The Data Operations Hub (DOH) is responsible for ingesting, validating, transforming, securing, and provisioning enterprise data for analytics, reporting, operational efficiency, and compliance.

Objectives to be fulfilled by the candidate:

As a Software Developer (Java), you will be a key member of the Data Operations Hub Team, designing and implementing analytical data solutions aligned with agency goals, leveraging expertise in data architecture, modeling, migrations, and integration, and collaborating with cross-functional teams.

Job Responsibilities:

1. Represent the Data Operations Hub team in forums, advising on Data Solutions.
2. Design and maintain scalable data solutions, including data lakes and warehouses.
3. Collaborate with teams to ensure data products support business needs and decision-making.
4. Evaluate and adopt emerging data technologies.
5. Develop architectural models using SparkX or other tools.
6. Serve as a subject matter expert in specific areas.
7. Work with modern data processing technologies such as Python, Java, Airflow, using data mesh and data lake architectures.
8. Contribute to the data engineering community and promote best practices.
9. Engage in hands-on coding and design for production solutions.
10. Optimize system performance by resolving inefficiencies.
11. Design reusable data frameworks with new technologies.

Required Skills (ranked by importance):

1. Experience as a Data Engineer or similar role leading technical teams.
2. Hands-on experience in system design, application development, and operational stability.
3. Expertise in architecture disciplines and programming languages.
4. Deep knowledge of data architecture, modeling, integration, cloud data services, and industry trends.
5. Practical experience with AWS/Azure and data engineering disciplines.
6. Advanced experience in ETL/ELT, event processing, etc.
7. Proficiency in SQL, data warehousing, and cloud-native relational databases like Snowflake, Athena, Postgres.
8. Strong problem-solving and communication skills.
9. Ability to evaluate and recommend future technologies.

Preferred Skills:

• Business architecture knowledge and experience with architecture assessment frameworks.

Required Education:

Bachelor's or Master's degree in Computer Science or a related field.