Senior Solutions Architect - Big Data Analytics Resume
OBJECTIVE:
- Looking for Senior Big Data Technology Architect position / Big Data Technical Project Manager roles
SUMMARY:
- Has more than 21 years of progressive IT experience in large - scale complex end-to-end application software development, product development, technology operations, managing and liaising with multicultural, multi-location teams, Vendor Management, Budget Management, Project Management, Process Improvement, Change Management, Release Management.
- Around 5+ years experience in ETL, Business Intelligence / Data Warehousing and reporting using - SAP BW/BI, SAP BO, Cognos, Informatica, Greenplum, SQL Server - SSRS, SSIS, SSAS, Oracle, Teradata, Datastage, Tableau, RapidMiner. Working knowledge of SAP HANA integration to Hadoop using HANA smart data access connectivity tool
- Oversight over database management, data migrations, specification of production monitoring requirements for new products and features, cost-tracking for existing applications and services, infrastructure enhancements/changes to improve existing applications/services and/or reduce internal costs.
- 5+ years of Big Data experience using HADOOP ecosystem tools and frameworks. Experience in Mongo DB, Cassandra, Oozie, HIVE, PIG, HBASE, Sqoop, Mapreduce, PySpark, Tez, Spark, HDFS, Shell scripting, MYSQL, PYTHON, Java, RESTFul web-services, Kafka, Storm, Ranger, Kerberos. Experience with data analytics, data mining computational methods to parse, link and develop patterns out of enterprise and consumer data, machine learning algorithms. Experience in statistical analysis software R, Mahout and SAS. Working experience in the cloud with AWS, Azure and Big Insights Well versed in installing and managing distributons of Hadoop (Hortonworks, Pivotal, Cloudera ). Knowledge in performance troubleshooting and tuning Hadoop Clusters. Good knowledge of Hadoop cluster connectivity and security.
- Linux system administration knowledge, understanding of Storage, Filesystem, Disks, Mounts, Network Elements etc. Experience in designing Data lakes. PoC and Use cases around data ingestion, metadata management, audit trails, governance and data security. Familiar with Optimized Row Columnar, Parquet, Compression techniques, optimizing JOINs Has both Big Data and relational database experience and deep knowledge of data modelling, MDM, Governance, Data Quality Assurance, Digital Marketing, Predictive and Prescriptive Analytics, data organization and storage technology, experience with high volumes and able to architect and implement multi-tier solutions using right technology in each tier, based on fit. Data modelling using ERWin tool.
- Around 7 years of Infrastructure and Telecommunication Systems support experience in implementing AMDOCS-ENSEMBLE, Enabler, OSS/BSS, CRM, Ordering and Billing at Client Sites in Germany, Cyprus, Israel, USA, Australia, India and Canada. Working experience on Ericsson Billing Systems. Thorough knowledge and experience with business requirements gathering & analysis, gap assessment, development of functional and technical specifications, data modeling, loads scheduling, testing, Reports development and user and support.
- Active team player and leader with excellent interpersonal skills, analytical skills, and effective written & oral communication and presentation skills.. Took many customer facing roles in building a dynamic work relation with client with continued focus on delivery and customer satisfaction.
- Experience Software Version management and Quality Assurance work. Experience in designing, developing and executing test strategies, test plans and test cases. Strong analytical skills with detail follow-through and problem solving skills in business process mapping and analysis. 4 years experience in Agile environment. Experience in managing offshore and onsite team in Agile environment in creating and managing user-stories and tracking progress, identifying bottleneck and addressing them Extensive experience working with business users as well as senior management in Retail, Telecom, Financial, Banking and Healthcare industry
TECHNICAL SKILLS:
Operating Systems: UNIX/LINUX, Sun Solaris, AIX, MVS, Windows 2000/NT/XP/98/95
RDBMS: Oracle 8i, 9i, 10g, 11g, MS SQL, MYSQL,M EMSQL Server 2005, DB2
NoSQL: Hbase, MongoDB, Cassendra, Couchbase
OLAP Tools: OBIEE 10g/11g, SAP BO, SAP Business Explorer (BEx) Analyzer, BEx Browser, Web reporting
ETL / ERP Tools: Informatica 7.1/8.x/9.x, Teradata, Cognos, SAP HANA, SAP ECC 6.0, SAP BI/BO, SAP Info Steward
BI Tools: OBIEE, Crystal Reports, SAP Business Explorer (BEx) Analyzer, BEx Browser, Web reporting
Languages: SQL, PL/SQL, JAVA, Shell Scripts, Python, R, C, C++, Pro* C, COBOL, Grunt Shell, Scala
Internet Programming: JavaScript, VBScript, HTML, and DHTML
Big Data Technology: Cloudera, Hortonworks Distributions,Pivotal, Sqoop, Pig, Hive, HDFS, MapReduce, Java, Impala, Hcat, HUE, flume, Spark, Hbase,Oozie, Falcon, Ambari, Zookeeper, Kafka, Asterdata, Vertica, Accumulo, Chef, Puppet,Alluxio, Polybase, Ceph,Atlas, Ranger, Cassendra, MongoDB, Nifi
Other Tools: Amdocs Ensemble, Amdocs Document Designer, OSS/BSS, Online charging, Enabler, AES 7.0, Ericsson Billing, Amazon Web Service, S3, EMR, SQL*LOADER, TOAD, PL/SQL Developer, ClearCase/Quest, HP Quality Center, XML, MS Project, Visio, TOAD, TEST DIRECTOR, WIN RUNNER, Dimensions, ChangeSynergy, ServiceDesk, GIT, MAVEN, Harvest, Testbase, Relational ClearQuest, Relational Quality Management, working knowledge of UML, XML and JCL, Gradle, Maven, Jenkins, Control M, Remedy
PROFESSIONAL EXPERIENCE:
Confidential
Senior Solutions Architect - Big Data analytics
Responsibilities:
- Data Governance/Catalogue/Self Serve tools evaluation with Apache Atlas, Solr, Elastic search, AWS Glue and SAP Information Steward
- Options to decouple storage and compute with Alluxio, Ceph, Polybase and Amazon S3
- Architecture road map for integrating current keystone platform with Hadoop for data analytics, Self Serve and eDiscovery
- Define and apply enterprise level standards and influence business strategy
- Coordinate solutions from technical perspective and minimize technical risk
- Clarify functional and non-functional requirements with BAs and EY Practitioners
- TPCDS performance benchmarking of BIGSQL with 10x100 GB load in 6-node BigInsights cluster in Azure cloud and present the results with IT and Business stakeholders
- Information Security to ensure compliance with secure development standards and negotiate security trade-offs for data in rest and data in motion
- Use Restful APIs to trigger Rapid Miner workflows to ingest data into BIGSQL and SQL server from various data sources
- Operations to take into account operational (non-functional) requirement
- Transferred the analyzed data across relational database from HDFS using Sqoop enabling BI team to visualize analytics.
- Real Time data ingestions PoCs into cold storage ( BIG SQL) and warm storage (SQL server ) using Kafka and Spark streaming
- Develop workflows for ETL and Data Science using Rapid Miner to generate the risk indicators and risk scores. Calculate composite risk scores on daily basis and generate alerts for Insider Threat and Bankers Supervision Program.
- ETL to Hive/BigSQL/SQLServer, data architecture, data modelling, data analysis, data profiling, metadata management
- Design Logging and Audit Trails framework for RapidMiner ETL and Data Science jobs
- Provide Multi-Tenant architecture solution for different client engagement using Ranger security features
- Well-versed with the client's chosen mode of project delivery, SAFE Agile and the related tools (TFS).
- Installation of HDP 2.6 cluster on Azure cloud using RHEL VMs with Ranger, BigSQL, Kerberize the cluster, AD/LDAP domain integration and user sync using SAMBA, WINBIND
- S3 object store integration with HDP 2.6.2, Atlas and Ranger
- Data in transit protection solution using the standard SSL-TLS 1.2 encryption mechanism
- Data at rest protection solution using the Vormetric Encryption mechanism on Microsoft SQL Server Storage file system. For all backups to Azure blob storage TDE encryption mechanism is used
- Configuration and Testing of Ranger/Sentry authorization policies on shared Blob storage, HDFS, Hive and BigSQL.
Environment: Azure - Big Insights, AWS cloud, HDP 2.6, RapidMiner, Multi-Tenant architecture, Tableau, BIGSQL, Kafka, NiFi, Tez, Spark, Python, JAVA, TFS, Hadoop, HDFS, Map Reduce, Linux, Hive, Spark, Mongo DB, Big Data eco system tools, PERL,Scala, Python, Falcon, YARN, AngularJS, Ranger, Dot Net, Kerberos, Ranger, HDP 2.6, Zeppelin, Polybase, MSSQL server upgrade from, Atlas, Solr, Elastic Search, AWS Glue, SAP Information Steward, BODS, SAS Grid, Tableue
Confidential
Architect - Big Data analytics
Responsibilities:
- Design schemas and create HIVE tables for importing data from multiple data sources using sqoop using dynamic trigger mechanism
- Designed solution for loading offers data created in Instant Savings in real time into Hadoop using RESTFUL APIs, SOAPUIs and oozie web services
- Involved in processing the data in the Hive tables using HQL high-performance, low-latency queries on Tez.
- Transferred the analyzed data across relational database from HDFS using Sqoop enabling BI team to visualize analytics.
- Processed hive tables using Spark hiveContext.
- Created and populated bucketed tables in Hive to allow for faster mapside joins and for more efficient jobs and more efficient sampling. Also performed partitioning of data to optimize Hive queries
- Design Offer Assignment scalability solution of Super node to process 60 million personalized data where we moved the OFFER ASSIGNMENT algorithm in Sams Personalize Engine (SPE) from hadoop cluster to Super node where we achieved 4X performance improvement
- Design/ Architect/ Implement various solutions arising out of the large data processing (GB’s/ PB’s) environment
- Real Time Load data from mainframe system to Oracle and Hadoop Backend using APIs, Web Services, SOAP and/or REST services and vice versa
- Apply other HDFS formats and structure (Avro, Optimized Row Column, Parquet) to support fast retrieval of data, user analytics and analysis.
- Implemented automation to reduce manual administrative tasks through use of jobs and scripts in Big Data migration areas ( Customer Knowledge Platform, Member Engagement and Campaign Management )
- Handling data ingestion process using generic scripts / falcon / oozie jobs that obtains the metadata/parameters necessary to bring the data in from the landing area of the edge node.
- Software Testing & Quality Assurance with AWS Lambda functions, SOAPUI
- Support business to complete Campaigns runs in production for Mass/ External campaigns and resolve production issues in timely manner.
- Data Ingestion from Model Centralization, Governance, Data Quality Assurance, Digital Marketing, and Predictive analysis
- Identified areas of improvement in the analytical models like Offer Assignment, Backfill and Offer Load
- Designed implementation strategies for improving overall efficacy of the critical processes in Offer Assignment and Backfill Models.
- Provided technical guidance to the team based on the application specific know-how and customer expectations.
- DevOps integration using CI/CD model using Jenkins
- Well-versed with the client's chosen mode of project delivery, SAFE Agile and the related tools (JIRA,Teamforge,Rally).
- Working experience on AWS - Kinesis, Firehose, Lambda, S3, Redshift, cloud formation, cloud watch, API Gateway
Environment: Cloudera, Pivotal and Hortonworks distribution of Hadoop. ORACLE, IBM-AIX/, Linux, Teradata, DataStage, Netezza, HIVE, Pig, HBASE, Scoop, Impala, Flume, Kafka, NiFi, Tez, Storm, Python, JAVA, GIT, Jenkins, Maven, SVN, Hadoop, HDFS, Map Reduce, Linux, Hive, Big Data, GIT, Stash, Spark, Mongo DB, Big Data eco system tools, PERL,Scala, Python, Falcon, YARN, AngularJS, Ranger, JAVA, AWS
Confidential
Technical Lead/ Solutions Architect - Revenue Management/ EDW
Responsibilities:
- Instituted stability and reliability program across IT including planning, execution, reporting and discussing findings and remediation plan with SLT, on applications, service architecture
- Improved billing SLA by one day and reduced system outages.
- Developed and implemented a long-term survival roadmap for legacy applications enhancement and infrastructure projects according to business projections and customer experience. Contributed within various governance bodies to develop IT wide standards/policies and processes.
- Managed relationship with all vendors in the environment.
- Remediated PCI, SOX compliance, securities and vulnerabilities findings.
- Attained improved application support and operational efficiencies by process enhancements
- Successfully designed, developed and Implemented End to End Billing Optimization project to reduce the run time of billing maps by 40%
- Monitoring batch jobs and provided solutions to systems performance issues
- Investigated production failures and provided resolution to these issues. Diagnosed and debugged issues.
- Solutioning and implementing with Amdocs for Rogers Long Time Survival projects for Server upgradation from UNIX to LINUX, ORACLE upgradation from 9i to 11g and application porting to the new Linux platform
- Resolved issues relating to performance, scalability, capacity and reliability
- Assisted in the development of the annual operating plan, forecasts, and budgets for Opex / Capex
- Knowledge of data acquisition from consumer mobile, web, STB devices and from internal and external service operator
- Worked with the Infrastructure and Network and cross functional teams for data architecture designs and integrations
- Collaborates with Infrastructure and Environment Support teams for seamless deployment
- QA and testing of modules/applications/interfaces.
- Preparation of documentation of data architecture, designs and implemented code
- Provide input on operational efficiencies to streamline and optimize platform operations.
- Knowledge of software Methodologies for Converged Networks and Services
Big Data Consultant
Confidential
Responsibilities:
- Big Data technologies related Proof of concept, proof of value, requirements gathering for use case, scoping, project planning, Hadoop cluster installations
- Harvested server logs and filtered error logs for processing in different stream.
- Ingestion of millions of CDRs data generated every second using Flume and real time data analytics using Storm
- Dropped call analysis and data anomality
- Experience in HDFS upgradation, commisioning, decommisioning of nodes, load balancing of clusters, monitoring and performance optimizing in Apache
- Diagnose, assess, troubleshoot and fix issues within the Open Source environment.
- Documentation of all environmental settings and configurations.
- Planning and upgrading of the environments both hardware and software .
- Ensure platform decisions result in high availability and stability of the systems in production.
- Did Linux Scripting, Pig Scripts, Hive queries.
- Data transformation and loading using pig, Hive
- Did design/deploy Hadoop Infrastructure solutions (high availability, big data clusters, elastic load tolerance)
- Resolved issues relating to performance, scalability, capacity and reliability
- Troubleshoot and debug Hadoop ecosystem run-time issues
- Importing and exporting data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa. troubleshoot production cluster issues, proactively monitor and measure cluster utilization and make recommendation
- Loading data into HDFS. Developing MapReduce Program to format unstructured data to structured data
- Proof of concept use case with anomality detection in CDRs, network switches
- Hands on experience with AWS - EC2, EBS, EMR, Kinesis, AWS Lambda and S3 file systems. Experience on data movements from data center to Cloud
- Hands on experience in loading data from UNIX file system to HDFS. Also performed parallel transfer of data from landing zone to the HDFS file system using DistCp.
Environment: HP-UX (Itanium)/ ORACLE, IBM-AIX/ORACLE, Cognos, SAS, WebFocus, ReportEase, Mapinfo, Essbase, Erwin, Netezza, Oracle, SQL Server, DataStage, Teradata, Cloudera distribution of Hadoop, Core JAVA, Hadoop, HDFS, Map Reduce, Linux, Hive, PIG, Big Data, MSSQL, GIT, Microstrategy, Asterdata, Vertica,,Scala, Mongo DB, Chef, Puppet
Confidential
Billing Solutions Prime/ Designer - Revenue Management/ EDW
Responsibilities:
- Participated in the project from initial blue printing, requirements gathering, transform business functional requirements into technical data model and query specifications.
- Configuration of the systems, testing and end user, writing functional design specifications for extracting and reporting.
- Write test schedule and test scenarios for integration testing, update technical documentation for processes and procedures
- Partitioned the cubes logically, physically. Partitioned ODS objects using oracle partitioning.
- Created Multi cube based on Sales order and Billing cubes.
- Working directly with business analysts, end users and project teams to develop new Business Intelligence functionality from specifications
- Converting complex requirements into an optimal design, BW Data Modelling. Developing reports using Tableau
- Coordinating testing, and performing change management /
- Monitoring, communicating and troubleshooting issues with data integrity, data design, and functional and technical software issues
- Preparing project-related documentation (object designs, business rules, technical specifications, etc.) throughout various stages of the project
- Participating in user acceptance testing (including scenario development), and documenting test results
- Data loading, creating aggregates, SQL queries performance tuning
- Write Linux/UNIX shell scripts (BASH) to execute ETL processes that ingest, transform, and publish highly refined analytical data sets
- Experience with Unix Internals, System Calls, IPC
- Master data management
Confidential
SAP BW Functional Consultant
Responsibilities:
- Performing analysis and configuration in SAP BW environment to facilitate various analytics solutions
- Working directly with business analysts, end users and project teams to develop new Business Intelligence functionality from specifications
- Converting complex requirements into an optimal design, BW Data Modelling
- Developing reports using BEx toolset - BEx Query Designer and Web Application Designer
- Coordinating testing, and performing change management /
- Monitoring, communicating and troubleshooting issues with data integrity, data design, and functional and technical software issues
- Preparing project-related documentation (object designs, business rules, technical specifications, etc.) throughout various stages of the project architected data warehousing using Kimball methodology
Environment: UNIX/ Oracle, SQL Server, DataStage, Informatica, Core JAVA, ECC 6.0, DAP BW/BI, MSSQL, TSYS, Banking domain, Loan / Mortgages, Agile methodologies
Confidential
Senior Solution Analyst
Responsibilities:
- Involved all phases of a project including Analysis, Design, Development, and Testing, Development of solutions, as well as Documentation and end-user
- Spearheaded conversion requirement, cost estimation, development, testing, implementation and post-production support phases.
- Executed end-to-end delivery of conversion activities for voice & data usages, customer management, financials, number management, interfaces and reporting.
- Debugged issues/defects, prioritized and rectified them.
- Co-ordinated in multi vendor environments with cross-functional teams to understand requirements, existing processes and practices in order to introduce right solution.
- Managed and assigned work to onsite and offshore teams in direct and matrix reporting.
Confidential
Team Leader
Responsibilities:
- Responsible for leading a development team of 3 people.
- Did coding of the paper and non-paper formats. Code Inspections, documentation of Impact Assessment, Specifications, Mapping documents.
- Design test cases, create test environment for testing, validation. My role was also to handle interface issues with other subsystems - Customer Support Management, Message Processing System, Billing and Print Shops.
Confidential
Programmer/Systems Analyst
Responsibilities:
- Liaison between client, onsite and offshore teams.
- Manage requirements, discuss solution, develop, implement and provide support.
- Impart to the user departments.
- Successfully delivered several solutions in production.
- Worked with several clients at the District and State and National level.
- Received several recognitions from District administration for successfully implementing Public Distribution System in Jorhat district.