We are looking for a Lead, Data Engineer who will oversee all the activities of the Data Engineering team. As the Lead, you will apply your extensive knowledge to manage designing, development, deployment and defining of best practices for engineering scalable secure data pipelines. You will collaborate with Data Engineers, Data Scientists, SW Engineers and UI/UX designers. You should have the ability to envision future improvements; you will lead the team to solve difficult challenges and set the pace.
This role provides an opportunity to work closely with the Executive Team at one of the fastest-growing Enterprise SaaS companies. The role is great for people who can work independently and are self-driven.
- Designing and implementing our data processing pipelines for different kinds of data sources, formats and content for the Near Platform.
- Participate in product design and development activities supporting Near’s suite of products.
- Developing techniques to analyze and enhance both structured and unstructured data.
- Develop/operationalize data science models in application contexts.
- Write and run comprehensive integration tests to deliver high-quality products.
- Involve in code and design reviews of different application modules.
- Capacity planning and specification of resource requirements for different deployment scenarios.
- Troubleshoot and provide a fix for any production bug on time.
- Gather feedback from stakeholders for improvement in code-stack and feature-sets.
- Assist the documentation team in providing good customer documentation, support and deployment guides.
- Adopt agile practices to track and update the status of assigned tasks/stories.
- Oversee and guide all activities of the junior Data Engineers.
Skills and Requirements
- You should hold a degree in B.Tech/M. Tech and have 8-10 years of experience with at least 5 years of working experience in any data-driven company/platform.
- Large scale distributed systems experience is preferred.
- Hands-on experience with Apache Spark.
- Good experience with Hadoop & big data technologies like HDFS, HBase, Apache Flink.
- Hands-on experience in Java.
- Experience preferred on data processing (both Batch & Stream Processing), large scale data storage (data lakes as well as stores)
- Must have experience with Cloud computing platforms like AWS/Azure/GCP.
- Ability to work independently with high energy, enthusiasm and persistence.
- You should have exposure in handling multiple simultaneous projects and meet deadlines and can work in a group setting as well as in an independent position.