Senior Big Data Engineer Job Vacancy in Vanya IT Noida, Uttar Pradesh – Updated today
Are you looking for a New Job or Looking for better opportunities?
We got a New Job Opening for
Full Details :
Company Name : Vanya IT
Location : Noida, Uttar Pradesh
Position :
Job Description : Job Title: Senior Data EngineerLocation: Noida (WFH)OverviewThis position is for Big Data Engineer specialized in Hadoop, Spark, Python, and AWS Data Engineering technologies with 3 to 8 years of experience.Roles & ResponsibilityFor this role, we require someone with strong product design sense. The position requires one to work on complex technical projects and closely work with peers in an innovative and fast-paced environment.· Grow our analytics capabilities with faster, more reliable data pipelines, and better tools, handling petabytes of data every day.· Brainstorm and create new platforms and migrate the existing ones to AWS, that can help in our quest to make data available to cluster users in all shapes and forms, with low latency and horizontal scalability.· Make changes to our data platform, refactoring/redesigning as needed and diagnosing any problems across the entire technical stack.· Design and develop a real-time events pipeline for Data ingestion for real-time dash-boarding.· Develop complex and efficient functions to transform raw data sources into powerful, reliable components of our data lake.· Design & implement new components and various emerging technologies in AWS, and Hadoop Eco System, and successful execution of various projects.· Optimize and improve existing features or data processes for performance and stability.· Conduct peer design and code reviews.· Write unit tests and support continuous integration.· Be obsessed about quality and ensure minimal production downtimes.· Mentor peers, share information and knowledge and help build a great team.· Monitor job performances, file system/disk-space management, cluster & database connectivity, log files, management of backup/security and troubleshooting various user issues.· Collaborate with various cross-functional teams: infrastructure, network, database.Desired Skills· Fluent with data structures, algorithms and design patterns.· Strong hands-on experience with Hadoop, MapReduce, Hive, Spark.· Excellent programming/debugging skills in Java/Scala.· Experience with any scripting language such as Python, Bash etc.· Good to have experience of working with No SQL databases like HBase, Cassandra.· Experience with BI Tools like AWS QuickSight, Dashboarding and Metrics.· Hands on programming experience with multithreaded applications.· Good to have experience in Database, SQL, messaging queues like Kafka.· Good to have experience in developing streaming applications eg Spark Streaming, Flink, Storm, etc.· Good to have experience with AWS and cloud technologies such as S3.· Experience with caching architectures like Redis, Memcached etc.· Memory optimization and GC tuning.· Experience with profiling and performance optimizations.· Experience with agile development methodologies and DevOps action.Job Types: Full-time, Regular / PermanentSalary: ₹526,829.00 – ₹3,800,000.00 per yearSchedule:Day shift
This post is listed Under Technology
Disclaimer : Hugeshout works to publish latest job info only and is no where responsible for any errors. Users must Research on their own before joining any company