Big Data Solution Architect Resume
3.00/5 (Submit Your Rating)
Salem, NH
PROFESSIONAL SUMMARY
- 11+ years of total professional experience in Design, Development of Applications wif 4+ years of experience in Big Data technologies wif frameworks and tools like Apache Spark, Apache Kafka, Apache NIFI, Apache Cassandra, Apache HBase, Apache Hive, ElasticSearch/Kibana/LogStash, Grafana/Graphaware, Zeppelin, Splunk, Sqoop, Pig, Zoo Keeper, Oozie, Flume and many other big data ecosystem tools.
- Extensively worked on solution architecture and design for Data lake analytics platform and executed / explored multiple pilots and implementations wif Azure tools and Google Cloud Platform tools like HDInsight, Stream Analytics, Data Lake Analytics, EventHub/IOT Hub, Datalake Store, Cognitive Analytics, Google Cloud Dataflow, Apache Beam Framework, Google BigQuery, Cloud PubSub, Cloud BigTable and etc.
- Extensively worked in performance tuning a massive data lake load process using very efficient design patterns for high throughput and low latency. Able to efficiently load 1 Billion records in less than 30 minutes.
- Currently working on Confidential and Confidential streaming frameworks extensively using both Scala and Python as teh main programming languages.
- Used Confidential Dataframes, Spark - SQL and RDD API of Confidential for performing various data transformations and dataset building and extensively worked on Confidential Streaming and Apache Kafka to fetch live streaming data.
- Good Knowledge on Cloudera/Hortonworks distributions and in Amazon simple storage service (Amazon S3), Amazon EC2, Amazon EMR and has very good understanding of Microsoft Azure and Google Cloud Dataflow Big data and machine learning tools.
- Extensively worked on loading data into Hive Tables, Raw HDFS Storage, Cassandra and Elastic Search from Using Confidential Jobs
- Good expertise in web log analytics using SPLUNK, ElasticSearch / Kibana and Grafana.
- Good experience in performing data analytics using Confidential wif both Scala and Python API’s.
- Strong experience in working wif UNIX/LINUX environments
- Implemented custom business logic and performed join optimization.
- Extensive experience wif leveraging HiveContext using SparkSQL to perform analytics on structured data.
- Experience in performing data modelling designs using HIVE partitioning/bucketing also leveraging dynamic partitioning
- Having experience in SOA, Web Services and REST API’s.
- Experience working in environments using Agile (SCRUM) and Waterfall methodologies.
- Defining Test Cases, analyzing bugs, interaction wif team members in fixing errors, Unit testing and User Acceptance Testing (UAT).
- Experience in creating and maintaining teh requirements definition documents that included Business requirements and Functional requirements.
- Exceptional problem solving skills and ability to quickly adapt to new challenges.
- Good understanding of Object Oriented Programming (OOPs) and very much experienced in scripting in Python and Scala.
- Developed UDFs in Python as and when necessary to use in PIG and HIVE queries.
- Extensive experience wif RDBMS databases - Oracle/MySQL/DB2 and No SQL databases - Cassandra and HBase.
- Extracted teh data from multiple sources into HDFS using Sqoop and Flume.
- Developed Hive scripts for end user / analyst requirements to perform ad hoc analysis
- Very good understanding of Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance.
- Developed Oozie workflow for scheduling and orchestrating teh ETL process.
- 2+ Years of experience as an ETL Developer/Analyst using Informatica power center express.
- 5+ years of Systems Analyst/Programmer experience in Mainframe Technologies for both Online and Batch Applications.
- Good experience in Extraction, Transformation and Loading data from various sources into Data Warehouses and Data Marts using Informatica Power Center (Repository Manager, Designer, Workflow Manager, Workflow Monitor, Metadata Manger), Power Exchange, Power Connect as ETL tool on Oracle, DB2 and SQL Server Databases.
- Creating mappings and workflow using Informatica power center express to load teh data to test databases/files for data analysis.
- Expert level skills in Stored Procedures development for ZOS/MVS Applications.
- Involved extensively in many Automation and SQL performance tuning for mainframe applications.
- Extensive experience writing CICS programs and specialist in writing efficient SQL queries.
- Played mainframe senior developer/developer roles in my early stages of my career.
- Good skills and knowledge of servant leadership, facilitation, situational awareness, conflict resolution, continual improvement, empowerment, and increasing transparency
- Strong Scrum Master experience and leadership experience wifin Agile environment.
- Expert level skills in facilitating Sprint Planning, Daily Scrums, Sprint Reviews and Retrospective Meetings.
- Good experience in creating teh Task Board and Sprint Burn down Chart at teh start of every Sprint using JIRA.
- Expert level skills in making teh Team aware of impediments and facilitate efforts to resolve them.
- Protect team from over-commitment, manage backlog, and prioritize resolution of defects/bugs as evidenced by teh on-time delivery of major initiatives.
- Assisted team wif making appropriate commitments through story selection, sizing and task definition and participated proactively in developing and maintaining team standards, tools and best practices reducing development time.
- Communicate wif other management, engineers, product managers and support specialists on product issues.
- Frequently create forum for communicating vision, goals, and Product Backlog items to teh team.
- Communicated TEMPeffectively across diverse audiences wifin and outside of teh Sprint Team (Stakeholders, Executives).
- Continuously provided perspective to teh team and keeps team focused on critical deliverables and tasks.
PROFESSIONAL EXPERIENCE
Confidential, Salem, NH
Big Data Solution Architect
Responsibilities:
- Extensively worked on solution architecture and design for Data lake analytics platform and executed / explored multiple pilots and implementations wif Azure tools and Google Cloud Platform tools like HDInsight, Stream Analytics, Data Lake Analytics, EventHub/IOT Hub, Datalake Store, Cognitive Analytics, Google Cloud Dataflow, Apache Beam Framework, Google BigQuery, Cloud PubSub, Cloud BigTable and etc.
- Extensively worked in performance tuning a massive data lake load process using very efficient design patterns for high throughput and low latency. Able to efficiently load 1 Billion records in less than 30 minutes.
- Expertise in designing and deployment of data lake wif different Big Data ecosystem tools including Spark, Kafka, Python, NIFI, Hive, Oozie, Sqoop wif Hortonworks distribution.
- Developed Confidential code using Scala and Spark-SQL for large data sets for both Streaming batch processing.
- Extensively worked on data model design for Hive Tables and Cassandra for low latency reporting
- Expertise in using various Confidential connectors to load and process data between Cassandra, Elastic Search, Kafka
- Extensively worked on loading data into HIVE tables using Spark.
- Extensively worked in Kafka and Confidential Streaming for unbounded API data, to perform various transformations, Joins and load into Elastic Search for low latency reporting and analytics.
- Leveraged to NIFI to configure for Streaming and Batch Sources for Pipelining into Kafka / HDFS Sinks
- Involved in converting Hive/HQL queries into Confidential transformations using Confidential RDD, Scala and Python.
- Developed SQOOP import utility to load data from various RDBMS sourcesfor history loads
- Developed data pipeline using Flume and Confidential to store data into HDFS.
- Good Knowledge on Cloudera/Hortonworks distributions and in Amazon simple storage service (Amazon S3), Amazon EC2, Amazon EMR and has very good understanding of Microsoft Azure and Google Cloud Dataflow Big data and machine learning tools.
- Extensively worked on loading data into Hive Tables, Raw HDFS Storage, Cassandra and Elastic Search from Using Confidential Jobs
- Implemented web log analytics using SPLUNK, ElasticSearch / Kibana and Grafana.
- Good experience in performing data analytics using Confidential wif both Scala and Python API’s.
- Strong experience in working wif UNIX/LINUX environments
- Implemented custom business logic and performed join optimization.
- Extensive experience wif leveraging HiveContext using SparkSQL to perform analytics on structured data.
Confidential, Tempe, AZ
Big Data Developer
Responsibilities:
- Provide in-depth technical and business knowledge to ensure efficient design, programming, implementation and on-going support for teh application.
- Worked extensively on Confidential using both python and Scala for data analysis.
- Good expertise in web log analytics using SPLUNK.
- Involved in writing optimized PIG Script along wif involved in developing and testing PIG Latin Scripts.
- Exported analyzed data to teh relational databases using Sqoop for visualization and to generate reports for teh BI team.
- Automated teh process for extraction of data from warehouses and weblogs into HIVE tables by developing workflows and coordinator jobs in Oozie.
- Logical implementation and interaction wif HBase.
- Used flume to collect teh entire web log from teh online ad-servers and push into HDFS.
- Load and transform large sets of structured, semi structured and unstructured data.
- Used Hive to analyze teh partitioned and bucketed data and compute various metrics for reporting.
- Installed Oozie workflow engine to run multiple Map Reduce jobs.
- Developed workflow in Oozie to automate teh tasks of loading teh data into HDFS and processing wif Pig.
- Creating Hive tables and working on them for data analysis in order to meet teh business requirements.
- Developed teh Pig UDF'S to pre-process teh data for analysis.
- Loaded data into HDFS and extracted teh data from Oracle into HDFS using Sqoop.
- Configured Flume to extract teh data from teh web server output files to load into HDFS.
- Facilitate Sprint Planning, Daily Scrums, Sprint Reviews and Retrospective Meetings.
- Create teh Task Board and Sprint Burn down Chart at teh start of every Sprint.
- Make teh Team aware of impediments and facilitate efforts to resolve them.
- Serve as a coach and mentor to members of teh Team.
- Respectfully hold teh Team, Product Owner and Stakeholders accountable for their commitments.
- Project budgeting/budget forecasting based on teh L0 estimates given by teh development team.
- Assisted team wif making appropriate commitments through story selection, sizing and task definition and participated proactively in developing and maintaining team standards, tools and best practices reducing development time.
- Communicate wif other management, engineers, product managers and support specialists on product issues.
- Executed assurance activities to identify and resolve any issues wif teh delivery model as early as possible
- Frequently create forum for communicating vision, goals, and Product Backlog items to teh team.
- Collaborated wif onsite BAs, SMEs, Tech Leads, QA Leads, and BU customers for gathering requirements to prepare Functional Requirements Doc, High Level Design and Detail Level Design documents, design walkthroughs, code reviews for onsite and offshore Dev teams .
- Developing teh work breakdown structure for teh high level business requirements.
- Identifying resources, delegating tasks and satisfying teh resource requirements.
- Coordinating wif all teh stakeholders in all teh stages of teh project lifecycle for teh smooth operation of teh project.
- Responsible for teh overall quality and timeliness of teh deliverables.
- Identified weaknesses in QA Processes, Web testing, Selenium Automation. Suggested & implemented improvements.
- Actively assist scrum and Product Owner master for grooming teh user stories for specific sprint and breaking down teh stories.
Confidential
Senior Tech Lead/Developer/Scrum Master/Business Analyst
Responsibilities:
- Performing in different roles as a Scrum Master, Business Analyst, Technical Lead, Senior Developer
- Proficient in obtaining project requirements from user and manager, formulating teh requirements into design specs, preparing system specifications, assigning tasks to team members, and tracking teh progress.
- Expertise in teh Mainframe tools like TSO, ISPF/SDSF,VAGEN, Panvalet, Endeavor, Xpeditor, Abend-Aid
- Experienced in creating JCL and JCL PROCs using various JCL utilities like DFSORT, FILEAID, IEBCOPY, IEBGENER, IEBCOMPR and ICETOOL.
- Experienced in creating High Level Design, Detailed Design and Functional Requirement documents.
- Extensive knowledge in onsite offshore model of working.
- Attained immense experience in usage of mainframe DB2 tools - SPUFI, File Manager for Db2 & QMF.
- Strong skills in working wif various debugging tools in teh mainframe environment for trouble shooting using XPEDITOR (CICS/Batch), Debugger, CEDF and Trace Master.
- Attained production/UAT support experience via usage of supportive tools including JOBTRACK.
- Strong Experience wif CICS Transaction processing wif DB2 application including Map/Map set creation.
- Developed multiple Map Reduce jobs in Python for log data cleaning and preprocessing.
- Exported analyzed data to teh relational databases using Sqoop for visualization and to generate reports for teh BI team.
- Automated teh process for extraction of data from warehouses and weblogs into HIVE tables by developing workflows and coordinator jobs in Oozie.
- Strong experience in DB2 application like cursor (Declare, Open, Fetch ), SQL query optimization, Cursor using pointer functionality.
- Good experience in Cobol-VSAM application wif VSAM database like KSDS and ESDS cluster
- Identify and verify teh impact based on teh changes in downstream/upstream applications.
- Prepare and review teh Detailed design specification document
- Coordinating teh offshore activities and providing them guidance on day to day activities.
- Performing Unit Testing, Integration Testing, Regression Testing and Shakeout Testing.
- Real time bug fix support during acceptance and end-to-end testing.
- Involved in teh implementation of these changes in production by packaging teh application etc.
- Involved in warranty Support for teh releases
Confidential
Senior Mainframe Developer
Responsibilities:
- Advance knowledge of Understanding manual, automated testing and Performance Testing-In-depth knowledge of IBM mainframes MVS, COBOL, JCL, VSAM, CICS and DB2 & extensive knowledge of IBM Mainframe tools and techniques as well as Unit regression testing
- Involved in understanding teh client requirements and project functionalities for preparing Test Strategy and Traceability Matrix document
- Handle responsibilities of preparing detailed test cases by referring teh code & database for Batch jobs and CICS screen for Online.
- Performed analysis, Functional, Regression, Integration, End-End and System testing, data verification and validation in a batch and interactive IBM MVS Mainframe environment (TSO/ISPF, JCL)
- Strong Experience wif CICS Transaction processing wif DB2 application including Map/Map set creation.
- Strong experience in DB2 application like cursor (Declare, Open, Fetch ), SQL query optimization, Cursor using pointer functionality.
- Good experience in Cobol-VSAM application wif VSAM database like KSDS and ESDS cluster
- Developed new inbound/Outbound program in CICS Web Services Environment wif support of CICS Transaction Server 3.1.
- Collaborated wif Business users for requirement gathering for building Tableau reports perbusiness needs.
Confidential
Senior Mainframe Developer
Responsibilities:
- Handle responsibilities of preparing detailed test cases by referring teh code & database for Batch jobs and CICS screen for Online.
- Performed analysis, Functional, Regression, Integration, End-End and System testing, data verification and validation in a batch and interactive IBM MVS Mainframe environment (TSO/ISPF, JCL)
- Strong Experience wif CICS Transaction processing wif DB2 application including Map/Map set creation.
- Strong experience in DB2 application like cursor (Declare, Open, Fetch ), SQL query optimization, Cursor using pointer functionality.
- Good experience in Cobol-VSAM application wif VSAM database like KSDS and ESDS cluster
- Advance knowledge of Understanding manual, automated testing and Performance Testing-In-depth knowledge of IBM mainframes MVS, COBOL, JCL, VSAM, CICS and DB2 & extensive knowledge of IBM Mainframe tools and techniques as well as Unit regression testing
- Involved in understanding teh client requirements and project functionalities for preparing Test Strategy and Traceability Matrix document
- Developed new inbound/Outbound program in CICS Web Services Environment wif support of CICS Transaction Server 3.1.
- Collaborated wif Business users for requirement gathering for building Tableau reports perbusiness needs.
Confidential
Mainframe Developer
Responsibilities:
- Advance knowledge of Understanding manual, automated testing and Performance Testing-In-depth knowledge of IBM mainframes MVS, COBOL, JCL, VSAM, CICS and DB2 & extensive knowledge of IBM Mainframe tools and techniques as well as Unit regression testing
- Involved in understanding teh client requirements and project functionalities for preparing Test Strategy and Traceability Matrix document
- Handle responsibilities of preparing detailed test cases by referring teh code & database for Batch jobs and CICS screen for Online.
- Strong experience in DB2 application like cursor (Declare, Open, Fetch ), SQL query optimization, Cursor using pointer functionality.
- Good experience in Cobol-VSAM application wif VSAM database like KSDS and ESDS cluster
- Developed new inbound/Outbound program in CICS Web Services Environment wif support of CICS Transaction Server 3.1.
- Collaborated wif Business users for requirement gathering for building Tableau reports perbusiness needs.
