Big Data Consultant/ Technology Architect Resume
SUMMARY:
- 11 Years of IT experience in Banking and Insurance Industry.
- Strong exposure to IT software project management, requirement analysis, estimates, design, development, implementation, maintenance/support and Integration of software projects.
- 4.5 Years of work experience in Big Data - Hadoop (Spark, Scala, Python, Hive, Sqoop, Marklogic, MongoDB, Pig, HCatalog, MapReduce, Hue, Ambari etc ) for development projects.
- 2 Years of work experience in Core Java and Shell Scripting.
- 1 Year of work experience in Python Scripting.
- 6.5 Years of work experience in Mainframe (DB2 SQL) Development and Support projects.
- Worked on RFPs and POCs.
- Business requirements gathering and preparation of detailed specifications that follow project guidelines required to develop written programs, scripts etc.
- Worked on using Scala and Spark to perform several aggregation logics.
- Expert in writing HiveQL, NoSQL (Mark Logic & MongoDB) queries as per business requirement.
- Expertise in Writing, Execution & Performance tuning of SQL Queries.
- Expertise in creation of reports using joins, unions, partition & bucketing as per business requirement.
- Strong background in all phases of software development life cycle (SDLC).
- Experience in job scheduling, monitoring, identifying the issue and resolving.
- Experience in writing, execution and debugging of DB2 Stored Procedures.
- Exceptional communication skills with proficiency at grasping new technical concepts quickly and utilize the same in a productive manner.
TECHNICAL SKILLS:
Operating Systems: Linux, MS Windows
Hadoop/Big Data: Cloudera HDFS, Hive, Sqoop, MarkLogic, MongoDB, Spark, Pig, MapReduce, Hcatalog, Hbase, Storm, Kafka,Flume, HiveQL, NoSql, Java, Scala, Python, Shell Scripting
Databases: NoSQL( MarkLogic, MongoDB), Hive, SparkSQL, DB2, MySQL, PostgreSQL
Programming Languages: Java, Scala, Python, Shell Scripting, C++Relevant Project Experience
PROFESSIONAL EXPERIENCE:
Confidential
Big Data Consultant/ Technology ArchitectResponsibilities:
- Business requirements gathering and preparation of detailed specifications that follow project guidelines required to develop written programs/scripts.
- Experience in importing raw data and exporting the analyzed data to the relational databases using Sqoop for data to be available for Hadoop.
- Implemented Hive optimized joins to gather data from different sources and run ad-hoc queries on top of them.
- Performed data transformations using Spark Scala/Hive.
- Developed Spark Scripts/Objects in Scala to analyze the raw data, populate staging tables and store the refined data in partitioned tables in HDFS.
- Responsible for Build/develop/testing shared components that will be used across many modules.
- Worked extensively in design and development of business process using Scala-Spark, Python, Sqoop, Pig and Hive.
- Implemented Partitioning, Bucketing in Hive for better organization of the data.
- Wrote Hive/Pig UDF's to perform business logic operations at record level.
- Used Pig to do data transformations, event joins, filter and some pre-aggregations before storing the data onto HDFS.
Environment: Hortonworks Hadoop, Spark, Scala, Hive, Pig, Sqoop, Hcatalog, MapReduce, Ambari, HDFS, Python, Shell Script, Workbench Harvest, Cigwin, AutoSys, AWS, Oracle, MS SQL Server, DB2, Core Java
Confidential
Big Data Consultant/ Technology ArchitectResponsibilities:
- Requirement Analysis, estimation, design, coding and testing for critical & complex enhancements and new application
- Troubleshooting and resolving issues during SIT, UAT and production phase of the project..
- Design ingestion patterns on basis of the any data type like CSV, XML, JSON, Map Message, RDBMS etc.
- Leading team in all development and support activates.
- Coding, code review, Junit and system testing for ingestion project.
- Creation of complex XQuery in NoSQL database (Marklogic) as per business need.
- L3 Production support - work closely with team members understand critical production issues. Troubleshoot and come-up with optimal code fixes. Review code, build packages for the prod deployment.
- Automation of manual tasks using shell script and java.
Environment: Linux, MarkLogic, Storm, HDFS, Hive, GitHub, Eclipse, Putty, WinScp, Java, Mavan, Nexus, Jira
Confidential
Data AnalystResponsibilities:
- Provide inputs on solution architecture based on evaluation/understanding of solution alternatives, frameworks and products.
- Involved in creating Python, Sqoop, Hive, Pig scripts as per business requirement.
- Handled importing of data from various data sources such as RDBMS, ftp server, MVS-S3 storage etc.
- Responsible for support, troubleshooting of Hadoop stacks e.g. Hive, Pig, Sqoop.
- Experience in writing of Pig and Hive UDFs in Java..
- Performance tuning in HiveQL and Pig Latin.
- Experienced in managing and reviewing Hadoop log files.
- Exposure in end-to-end performance tuning of Hadoop clusters routines against very large data sets.
- Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers.
Environment: Linux, Cloudera HDFS, Hive, Hbase, Sqoop, Pig, MapReduce, Oozie, HiveQL, NoSql, Core Java
Confidential
Lead DeveloperResponsibilities:
- Requirement Analysis, estimation, design, coding and testing for critical & complex enhancements and new application.
- Providing support to team in understating and resolution of technical issues.
- Helping team in understating the new requirement.
- Leading team meeting with onsite/customer.
- Requirement Traceability Matrix Creation & Verification at various phases of testing.
- Preparing Weekly/Daily status reports.
- Lead the team member on the tasks progress.
- Involved in implementation support.
- Tracking team activities.
Confidential
Senior DeveloperResponsibilities:
- Analysis of complex production issues/tickets and resolution within the specified time as per SLA based on the severity levels.
- Mainframe job scheduling monitoring, identifying, resolving and escalating job failures, problems, application and system alerts that could impact the timely completion of batch schedules critical path.
- Impact analysis, estimation, design, coding and testing for critical & complex enhancements and new application.
- Providing support to team in understating and resolution of technical issues.
- Helping team in understating the new requirement and leading them for technical issues.
- Leading team meeting with onsite/customer.
Confidential
Responsibilities:
- Analysis, Estimates and Design phase of Confidential project. I was responsible for enhancement for Swift & RTGS, Cops, Manual input and Payment upload applications
- Coding, Unit testing, System and System Integrations testing of Confidential project for Swift & RTGS, Cops payments, Manual input and payment upload applications
- Creation of EOD & EOM account balance application. Responsible for creation of Design, Coding and all phases of Testing for this development project.
- Provided support to users in User Acceptance Testing.
- Execution of EOD & EOM batch in different test environments.
Confidential
Responsibilities:
- Requirement Analysis and estimation of the all development tasks.
- Analysis, Coding, Testing & enhancements of related application.
- Requirement traceability matrix creation & verification at various phases of testing.
- Preparing weekly Status reports.
- Conducting weekly status Meetings including Onsite.
- Lead the team member on the tasks progress.
- Involve in implementation support.
- Tracking team activities.
- Providing technical support to the team members.
- Initiatives on defect prevention activities.
- Documentation, i.e. technical, functional, query log, daily & weekly status reports.
Responsibilities:
- Analysis of new business requirement & details design document creation.
- Worked build-in phase e.g. Program specification, unit test plan creation, of programs, JCL, Procedure, Control card etc. Unit test cases execution & peer review.
- Handled production support activity for mainframe systems, activities includes problem analysis, fixing, testing and implementation to the production environment.
- Documentation, i.e. technical, functional, query log, daily & weekly status reports.
- Communicate with onside team/customer by conference calls and mailsAwards and Accolade
- Awarded by “Bravo” for performing key role in development and implementation of a critical release with excellent quality.
- Awarded by “Making a Difference” award for successfully executing a challenging project within client defined deadlines.
