Hadoop Data Analyst,integrated Fire/auto Data Capture Resume
Bloomington, IL
SUMMARY:
- Demonstrated expertise in the entire project and software development life cycle from initial requirements gathering through design, development, testing and deployment.
- Adept at communicating with diverse, cross - functional stakeholders to understand business needs to create sophisticated solutions using cloud, big data, and othercutting-edge technologies.
- Contribute to large-scale, mission-critical projects, including legacy system and data migration, application enhancements, and system rewrites.
- Perform hands-on coding, testing, debugging, root cause analysis, and problem resolution.
- Highly skilled in working with legacy mainframe databases, as well as mainframe development and programming.
TECHNICAL SKILLS:
Operating Systems: OS/390, MVS, Linux, Ubuntu, Windows, UNIX
Languages: COBOL, JCL, SQL, PL/I, SAS, Java, Easytrieve, Scala, Python, C, Assembler, Fortran, Pascal
Storage: DB2, VSAM, IMS, Supra, MSAccess, MySQL, Postgres, MongoDB, SQL Server, Cassandra
Framework: Spring MVC, JDBC, Struts, Spring Web Flow, Spring Boot, Spring IOC, Spring Data
Online: Mantis, CICS, IMS/DC
Scripting: JavaScript, Shell Scripting, PHP, REXX
Big Data: HDFS, MapReduce, Spark, Pig, Hive, HBase, SQOOP, Flume, Impala, Kafka, Oozie, Cloudera VM, CloudxLab Cluster, Hortonworks Sandbox,Solr
AWS: S3, EC2, EMR, EBS, VPC, Kinesis, Route-53, RDS, DynamoDB, Lambda, CloudWatch, CloudTrail, CloudFront
Web Technology: J2EE, JSP, XML, HTML, Servlets, EJB, JDBC, Applets, JSON, REST, SOAP, JQuery
Version Control: Revision Management System, GITHub, Subversion
Mainframe Tools: File Aid, File Manager, SPUFI, DB2Menu, FMDB2, BTS, DUG, Interactive Data Capture, BMCDB2, SFDEV, Register, Autoflow, QMF, BLDISS, BLDDOC, ISSTAT, MQUIRY, STAGEISS, DGA ISPF, Star Tool, Version Check, Star Tool, Xpeditor, Mainframe Express, CONTROL-M, Connect Direct, FTP
IDE: Spring Tool Suite(STS), Eclipse, WebSphere Studio Application Developer, Rational Application Developer, Enthought Canopy, IBM Data Studio, PyCharm
Build/Testing Tools: Maven, SBT, JUNIT, POSTMAN, SOAP UI
Print Tools: Advanced Function Presentation, CONTROL-D, Prinova, Ricoh Process Director, HP-Exstream
Change/Service Tools: HPService Manager, HP Service Desk
Methodologies: Agile, Waterfall, Test Driven Development
File Format: Text,Avro,Parquet,JSON, XML, Sequential and all Mainframe File Formats
File Transfer: Putty, WinScp, Scp, FileZilla
PROFESSIONAL EXPERIENCE:
Confidential, Bloomington, IL
Hadoop Data Analyst,Integrated Fire/Auto Data Capture
Responsibilities:
- Hadoop Data Analyst,Integrated Fire/Auto Data Capture - Streaming Data Play key role in streaming data phase of large-scale Enterprise Performance Management & Analytics (EPMA) data capture project.
- Analyzed requirements, performed coding, and executed testing.
- Assisted in creating Spark MapReduce program to extract data for analytics from raw XML data.
- Collaborated on creating source to target data mapping document.
- Used Spark streamlining API to perform transformations and actions to move data from Kafka in near-real-time and persists into HDFS.
- Scheduled Spark and Hadoop jobs using Oozie workflow engine.
- Ran hive queries to perform data analysis based on business requirements
- Used Spark Context, Spark-SQL, Data Frames, and Pair RDDs to optimize Hadoop algorithms.
- Hadoop Data Analyst,Integrated Fire/Auto Data Capture Participated in all phases of Big Data Implementation project, including requirements analysis, design, development, building, testing, and deployment of Hadoop cluster in fully distributed mode
- Mapped to RDBMS data types to Hive data types; performed data validation.
- Created MapReduce program to extract data for analytics from Edge Node raw data to HDFS.
- Executed Hadoop streaming jobs to process Flume-ingested terabytes of XML data.
- Scheduled Hadoop jobs and shell scripts using Oozie; created workflow in Oozie to manage jobs on Hadoop Cluster and trigger daily, weekly, and monthly batch cycles.
- Implemented proof-of-concept (POC) to migrate MapReduce programs to Spark Transformations using Spark and Scala.
- Hadoop Data Analyst,Auto Change Data for Analytics Gathered requirements and performed coding and testing for project to transform raw auto policy data into usable format.
- Developed MapReduce program to extract raw data, created optimized map joins to clean data prior to running algorithms, and executed queries in Hadoop and DB2 to verify data load.
- Hadoop Data Analyst,Move Filenet Data to Hadoop Performed requirements analysis, coding, and testing for implementation of Hadoop to replace IBM FileNet Image Services.
- Used SQOOP to import/export data into HDFS and Hive, copied data from Edge Node to Hadoop, and ran queries to verify data load.
- Wrote Pig UDF for data transformation and implemented oozie workflows for Scoop jobs and Pig UDFs
- Mainframe/Hadoop Tester, Legacy Modernization Performed testing for proof-of-concept to move data from legacy mainframe DB2 tables to Hadoop environment. Analyzed requirements, performed coding, and executed tests.
- Collaborated on creation of JCL using FTP to transfer data from mainframe to Edge Node on UNIX Servers and assisted in copying data from Edge Node to Hadoop; ran queries to verify completion of data transfer.
- Senior Systems Analyst, Store Documents in Customer Data Center Analyzed requirements for solution to enable electronic storage, viewing, and delivery of documents for customers, agents, and other staff. Performed coding, testing, technical support, and quality assurance.
- Designed and coded new programs using COBOL to create metadata for XML and print lines for PDF report generation on UNIX server.
- Analyzed, designed, and developed JCLs/JOBSTREAMS to transfer print line files from mainframe to UNIX.
- Spearheaded root cause analysis to identify and resolve issues in test and production environments during SIT, UAT, and implementation phases.
- Prepared data element mapping documents between COBOL copybooks and XML.
- Team Lead/Mainframe Developer, Bank Print Internal Application Member of project from requirements gathering through design, development, coordination, system testing and implementation, defect resolution, and production support.
- Performed project planning and resource identification/allocation to ensure optimal utilization, seamless execution, and on-time delivery.
- Created JCLs to send documents to UNIX, developed Web service interface using WebSphere studio to access data on mainframe DB2, and wrote stored procedures to pull customer information for batch printing and/or online retrieval.
- Team Lead/Mainframe Developer, SFPP Print ApplicationProject Collaborated on creating bills and cancellation notifications for customers using data batched daily from multiple tables. Assisted in converting application from print format to processing through HP Exstream.
- Adhered to standard change management and service management procedures to coordinate revision management and ensure interoperability of system and successful implementation of components.
- Team Lead/Mainframe Developer, Audit & Collection Notices ConversionProject Facilitated JAD sessions to review business document with end users, application developers, and technical analysts; prepared analysis and understanding documents.
- Led project resource planning to ensure effective allocation of resources for on-time delivery.
- Liaised with vendors to create templates for legal audit documents and audit/collection application to be generated through HP Extreme.
- Designed, coded, tested, debugged, documented, and supported applications in accordance with established specifications and business requirements to deliver product on schedule; provided 24x7x365 support for application availability, reliability, and stability.
- Senior Systems Analyst/Team Lead/Fire Print Application Fully responsible for requirements gathering, impact analysis, estimation, design, construction, code walkthroughs, documentation, testing, implementation, and production support for multiple concurrent projects for commercial and personal business lines
- Reduced system resource needs by identifying and resolving numerous inefficiencies. Debugged and maintained code for complex systems. Conducted root cause analysis and provided 24x7x365 support.
- Mainframe Developer, Conversion of EOD/IOD Led requirements gathering, impact analysis, estimation, design, construction, code walkthroughs, documentation, testing, implementation, and production support for automation of policy inserts to reduce vendor costs.
- Collaborated with forms design and legal teams to create page segments for inserts and endorsements. Worked with DBA to create DB2 tables and reviewed SQL code for different scenarios.