Big Data Architect Resume
DallaS
SUMMARY:
- 22 years of professional experience in building Enterprise applications in US.
- Experience in developing software solutions on Big Data Platforms in Hortonworks, Cloudera and AWS.
- Hands on development experience in Hadoop ecosystem components, Map Reduce, YARN, HDFS, HBase, Apache Hive, Apache Sqoop, Pig, Zookeeper, Oozie, SOLR, Flume
- Experience in cloud computing, Amazon Cloud services, Microsoft Azure, AWS, SaaS, IaaS and PaaS
- Expertise in big data application tuning and optimization
- Expertise in Scala, Python, Apache Spark RDD, Spark SQL, Spark Streaming, DataFrames and Kafka
- Familiarity with machine learning pattern using Mahout and Spark MLlib
- Expertise in Business Intelligence tools—Tableau, PowerBI and QlikView
- Expertise in NoSQL Database HBASE, Cassandra, Mongo DB and Apache Phoenix
- Experience in configuring cluster and installing the services and monitoring the cluster.
- Technical Leader with 15 years of experience in HIGH volume database design and architecture
- Expertise in gathering business requirements, capacity planning, resource planning, functional analysis, design, implementation and gap analysis.
- Hands on development and basic administration experience in Oracle 12c/11g/10g/9i/8i/7 RDBS server.
- Expertise in Conceptual, Logical and Physical design of both OLTP and OLAP (Data warehouse) databases.
- Very strong programming skills in SQL, ANSI SQL, PL/SQL, Database Packages/Procedures/Triggers, ERWIN, Oracle SQL Modeler, Oracle Designer, SQL*Loader, Data Pump and Data Base Administration tools.
- Expertise in high transaction database modeling, design and development.
- Sound knowledge in MySQL, Teradata, MS SQL server and ETL Tools (SAS, Data Stage and INFORMATICA).
- Expertise in building J2EE Enterprise applications for financial, medical, automobile, insurance, telemarketing, production planning, MRP II and human resources management industries.
- Experience in complete software development life cycle (SDLC) and software maintenance.
- Very strong programming skills in Java, Servlets, JDBC, JSP, HTML and JavaScript.
- Strong programming skills in EJB, Java/J2EE, Struts framework, Spring Frame Work and Spring MVC
- Sound knowledge in Hibernate, JPA and IBATIS.
- Hands on experience in WebSphere App Server 6.x /7.x, Web Logic 9.x/10.x, RAD 7.x/8.x and WSAD.
- Strong programming skills in Business Objects and Crystal XI.
- Sound knowledge in RUP and Agile methodology - SCRUM, Rational Tools and Eclipse IDE.
- Experience in onshore-off shore round the clock project development and maintenance model.
- Excellent written and verbal communication skills.
TECHNICAL SKILLS:
HDFS, Hive, Sqoop, PIG, Oozie, Flume, KAFKA, SOLR, Zookeeper, MAP Reduction, YARN, TEZ, HUE, Teradata SQL Assistant for Hadoop, IMPALA, Zeppelin, Ambari, SPARK, NoSQL, Dynamo DB, HBase, MongoDB, Apache Phoenix, Ambari, Cloudera Manager, AWS, EC2, S3, SaaS, IaaS, PaaS, Windows Azure, Oracle Data Base Server (12c/11g/10g/9i/8i/7/6)MySQL, Teradata, MS SQL Server, ANSI SQL, PL/SQL (All versions), TSQL, ERWIN, SQL data modeler, Oracle designer 2000/6i/11g, Oracle SQL developer 3.x, TOAD, SQL Navigator, Oracle JDeveloper, ETL Tools - SAS9.x, Data Stage 8.x,9.x, INFORMATICA 6.x, Java2, Java 5/6/7, SCALA, HTML, XML, UML, FpML, Java Script, JSP, Servlets, EJB 2.x, DTD, XSD, XSLT, AJAX Struts Frame Work, Apache Tiles, Spring Frame Work, Apache Commons, JSON, AVRO, Parquet, Hibernate, Design Patterns, ANT, Hudson, SOAP, WSDL, JAXB, JUnit, Eclipse 3.x,RAD6.x/7.x/8.x WSAD5.x, Change Man, PVCS, Visual Source Safe, Clear Case, IBM Web Sphere (6.x/ 7.x) BEA Web Logic (8.x/9.x,10.x), Tomcat,, MS Visio, Power Point, MS Project, MS Office products, Developer, Forms, Reports, SQL*Loader, Data Pump, Oracle Financials 10.7, 11i, Tableau, Qlikview, Business Objects, Crystal Enterprise XI, Rational Rose, MS Windows (95/98, NT/XP/Vista/7/10.x),Unix (HP-UX, Sun Solaris & AIX), Linux, PYTHON and Shell scripts
PROFESSIONAL EXPERIENCE:
Confidential, Dallas
Big Data Architect
Responsibilities:
- Define project scope, goals and project deliverables interacting with Confidential & Confidential Product Managers
- Responsible for the Design and Architecture of Hive Data warehouse for 1 PB database
- Heavy coding efforts to develop and test various Hadoop application components.
- Code and Test hundreds of hive programs using Python, hive HQL, Scala and Spark SQL
- Code dozens of SQOOP import to support data ingestion/ETL to Confidential & Confidential 790 Nodes Hadoop Cluster
- Create dozens of SQOOP export programs for customer interface into Oracle and SQL Server RDBMS
- Expertise in Scala, Spark SQL, Data Frames and Spark RDD
- Expertise in Hortonworks, Beeline, Linux Scripting, Kerberos, Kafka, Zookeeper, Ambari and Zeppelin
- Deploy hive, spark and python programs to PROD environment
- Expertise in HUE, HWX Hive View 2.x, Teradata SQL Assistant and TOAD for Hadoop
- Develop process automation using Oozie and crontab
- Create BI data visualization components using Microsoft PowerBI
- Hadoop application tuning of Hive and Spark program to optimize the latency and through put
- Interface Hive Data warehouse environment with NoSQL database
- Train and Mentor other team members in the group
- Support Map Reduction programs written SCALA and Hive HQL on the Hadoop cluster
Confidential, Dallas
Big Data Architect
Responsibilities:
- Perform data modeling and design for Chase Mortgage, HELOC and Auto loan applications
- Define and manage the architecture and life cycle of Hadoop and SPARK projects
- Develop Map Reduction jobs in Hadoop in Scala/Python/Java/PIG/Hive
- Expertise in writing Scala code to run SPARK jobs in Hadoop HDFS cluster
- Expertise in SPARK Streaming using KAFKA
- Expertise in SPARK SQL, DataFrames, RDD and MLlib
- Experience in writing Pig Latin scripts to load data in to HBase in AVRO format
- Hands on experience in Hive, HiveQL, Impala and Parquet
- Hands on experience in NoSQL Databases Mongo DB and HBase
- Hands on experience with data loading tools like Flume, Sqoop
- Manage Hadoop jobs using scheduler Oozie
- Cluster Coordination services through Zookeeper
- Expertise in Data visualization and Business intelligence using Tableau
- Create database design artifacts, ERD, DFD, conceptual, logical and physical designs using ERWIN for various databases, Oracle, MySQL and NoSQL HBase
- Design data warehouse for star schemas, dimensional, bottom-up Kimball approach
- Design architecture for ETL loads.
- Lead and hands-on effort in Develop, Test and Optimize about 300 Oracle APIs-business logic units in PL/SQL
- BMC Control M designs and implementation for project automation.
- Conversion of SQL Server database procedures into Oracle database packages in PL/SQL using MS visual studio and TOAD.
- Source code control of Oracle APIs, Database packages, Triggers, DDL, DML and DCL scripts in Subversion
- Implementation of projects using Agile SCRUM product development strategy.
- Mentor and train junior staff in the team.
- Demonstrated competence around the areas of product knowledge, relationship building, professionalism and personal excellence, leadership, people focus, and business development.
Confidential, Chicago
Hadoop Architect/ Solution Architect
Responsibilities:
- Participates in facilitating the definition of project scope, goals and project deliverables
- Responsible for managing the development and deployment of Hadoop application components
- Used Oozie to automate data loading into the Hadoop Distributed File System and PIG to process the data
- Provided design recommendations and thought leadership to upper management that improved processing speed and data storage problems by leaps and bounds
- Hands on experience to move data from RDBMS and data warehouse to Hadoop HDFS cluster using SQOOP
- Data visualization efforts against HDFS, Hive and NoSQL databases Mongo DB and Casandra
- Perform data modeling and analysis for NT Compliance Analyst and Liquidity applications
- Create data modeling artifacts using ERWIN for OLTP and OLAP (Data warehouse) projects
- Create detailed design for new projects. Generate DDL, DML, DCL scripts for both traditional and dimensional
- Create design artifacts and documentation in UML (ERD, DFDs, Sequence diagrams, Class Diagrams…)
- Design and create database tables with partitions to store billions of rows
- Code and Test several database packages with several stored procedures and functions
- Develop and test Data Stage (ETL) Jobs to extract data into data repository and Oracle database from upstream systems
- Written a number of shell scripts and parser routines to automate data extraction from several upstream systems Bloomberg, AIP, NDR, FpML, SWAPs, Options and OTC DataMart
- Work on project automation using Control M scheduler
- Expertise in Data visualization and Business intelligence using Tableau and Qlikview
Confidential, Chicago
Technical Lead/ Data Architect
Responsibilities:
- Design Big Data model to transfer data into Hadoop echo systems from Oracle and SaaS data stores as part of company’s new initiate data analytics
- Data visualization efforts against HDFS and Hive
- Written code to move data into HDFS cluster using SQOOP and PIG Latin
- Perform data modeling and analysis for a 5 Terabytes OLTP data model.
- Create database design with integrity constraints and triggers.
- Create design artifacts and documentation for OLTP and Data warehouse.
- Work with business stake holders and business analysts to create credit models.
- Design and create database tables with partitions to store billions of rows.
- Perform basic database administration on DEV and QA instances.
- Develop over 30 database packages with dozens of stored procedures and functions.
- Written several SQL*Loader script to migrate data from legacy databases.
- Written a number of shell scripts to automate data loads from CSV files.
- Write and optimize hundreds of SQL queries and database views for Java/J2EE UI interface.
- Design data warehouse for star schemas, Top-down Inmon methodology.
- Develop and implement data integration architecture with OLAP database using SAS.
- Develop technical detail design for all the major tasks.
- Assign tasks to team members onsite/offshore and monitor status till completion.
- Conduct project status and follow-up meetings and discussions.
- IBM application server WebSphere 7.x administration tasks.
- Perform root cause failure analysis and recommend resolutions.
- Mentor and guide junior team members onsite/offshore.
- Guide QA team to test the software components developed.
- Involved in unit testing, integration testing, feature testing and Regression testing.
- Participate in software gap analysis with business analysts and product managers.
- Manage open tickets and find resolutions.
Confidential, Chicago
Technical Lead/Solution Architect
Responsibilities:
- Work with business users and business analysts to develop requirements and translate requirements into technical designs.
- Functional analysis, data modeling and database design for Oracle 11g RDMS Databases.
- Data migration from legacy systems and other DBMS Systems.
- Research advanced concepts to develop reusable solutions to recurring problems.
- Work with product managers on capacity planning.
- Participate in software gap analysis with business analysts and product managers.
- Created several DB packages, stored procedures, functions, and triggers.
- Wrote large number of Oracle queries. Heavy coding effort in PL/SQL. Data base administration.
- Analyze functional requirements, Use Cases and technical specifications for J2EE Architecture and design.
- IBM application server WebSphere 6.x administration tasks.
- Preparation of design documents in UML.
- Project Management and sizing.
- Lead and mentor developers in the team
- Implementation of Struts MVC pattern frame work using Struts2.
- Unit testing using Junit.
- Involved in unit testing, integration testing, feature testing and Regression testing.
- UNIX and Shell scripts, Source control management in Subversion.
Confidential, Melrose Park, Chicago
Team Lead and Solution Architect
Responsibilities:
- Conduct sessions/interviews with the business side to secure detailed information pertaining to application and data usage.
- Worked both directly with business users and with business analysts to develop requirements and translate requirements into technical design
- Functional analysis, data modeling and database design for Oracle 11g RDMS Databases
- Preparation of design documents in UML
- Weekly communication and reporting of project status risks and issues
- Construct and optimize complex SQL queries
- Heavy coding effort in PL/SQL
- Assign tasks to team members onsite/offshore and monitor status till completion.
- Mentor and guide junior team members onsite/offshore.
- Guide QA team to test the software components developed.
- UI coding efforts in JSP, JavaScript and HTML
- Develop applications using Spring MVC pattern frame work.
- Integration with IBM MQ series
- Used SOA and Web services for importing BOM structure from TEAMCENTER application
- Develop WSDLs, SOAP.
- Heavy hands on programming efforts in control and UI layer.
Confidential, Chicago
Solution Architect
Responsibilities:
- Analyze functional requirements, Use cases and technical specifications.
- Application architecture and design.
- Preparation of design documents in UML.
- Research advanced concepts to develop reusable solutions to recurring problems.
- Construct and optimize complex SQL queries.
- Go to guy in the team for Oracle queries, stored procedures, functions, packages and triggers.
- Implementation of Spring MVC pattern frame work.
- Data management and persistent layer development in open source ORM tool IBATIS.
- Implementation of Dependency Injection using Light weight Spring frame work
- Heavy hands on server side programming efforts in Java.
- UI coding efforts in JSP, AJAX, JavaScript and HTML.
- Extensive coding efforts to develop oracle data base packages and IBATIS mapping files.
- Design and develop SSL encryption and single sign on features of the application.
- Rational tools Clear case and Clear quest for source control and defect tracking.
- Mentor junior programmers in the team.
Confidential, Chicago
Application Architect
Responsibilities:
- Data migration and data loading programs in Oracle environment
- Conversion of Oracle PL/SQL systems into J2EE Application.
- Written a number a Database packages and triggers in PL/SQL
- Data modeling and database design focused on Oracle environment
- Installation, administration and maintenance of Oracle 10G Database for the development and QA environment
- Construct and optimize complex SQL queries
- Analyze functional requirements, Use Cases and technical specifications
- Preparation of design documents in UML
- Implementation of Struts MVC pattern frame work using Action Servlets and JavaBeans
- Data management and Persistent layer development in open source ORM tool Hibernate
- Implementation of Dependency Injection and AOP using Light weight Spring Frame Work
- UI coding efforts in JSP, JavaScript and HTML
- Extensive programming efforts in server side Java business layer
- Rational tools Clear case and Clear Quest for version control and defect tracking
- Mentor junior programmers in the team, both onshore and offshore.
- Involved in unit testing, integration testing, feature testing and Regression testing
