SUMMARY

• Working as a Developer/Architect in Big Data solutions that involved Horton Works, Cloudera, Microsoft Azure, GCP and AWS
• Cloudera Certified Developer for Apache Hadoop with 21 Years of IT experience in Development, Design, Application Support & Maintenance and managing projects of IT applications.
• Experience in Planning and Defining Scope, Developing Schedules, Budgeting, Cost estimations, Team Leadership, Monitoring and Reporting Progress
• Experience in JAVA patterns by using Open - Source products.
• Experience in solutions involving end to end using Hadoop HDFS, Map/Reduce, Strom, Solr, Kafka, Scala, Pig, Hive, HBase, Sqoop, Oozie, Arow and performance tuning the Hadoop cluster on on-premise and cloud.
• Experience in programming python, Java and Scala on Spark
• Experience in real time streaming using Kafka and Storm.
• Experience in estimation of infrastructure and capacity planning of the cluster.
• Experience in Installing and configuring and upgrading Hadoop cluster using Cloudera Manger and deploy tools.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Experience working with large databases like Oracle, MySQL, Teradata, DB2, Postges and GreenPlum, Redshift
• Experience in data migration and data modeling using N0-SQL databases (HBase, Cassandra)
• Knowledge in Data analytics using R Lang, Apache Hadoop, Map-Reduce
• Knowledge in Data Analysis techniques like clustering, classification, regression, forecasting and prediction using R Lang
• Knowledge on BI and data warehouse processes and techniques
• Expert in working with multi-threaded applications using VC++ and C++ in Windows, UNIX and Solaris Environment
• Experience in various activities of Agile Methodology UMF, UML and Design patterns
• Experience in various phases of Software Development such as Study, Analysis, Development, Testing, Implementation and Maintenance of a Real-time systems.
• Experience in building applications on Kafka /Kinesis
• Knowledge on Data analytics, Data Science and Machine learning
• Experience in Managing and leading the projects with Onsite/Offshore model

TECHNICAL SKILLS

Programming Languages: Java, Python, R Lang, Map reduce, Pig-Latin, Scala, C++, VC++, C# .Net 4.0.

Tools: MS Project, Visual Studio, CVS, Sqoop, Oozie, zookeeper, Spark, Storm, Nifi, Kafka, AWSPowerShell, SVN, Bitbucket, GitLab, AIM, Tableau, Looker, AWS.

Frameworks: Hadoop HDFS (Apache, Cloudera, Hortonworks), Spark, Cassandra, Microsoft Azure HDInsight, MVC.

Database: My SQL, Oracle, Greenplum, Teradata, Hive, HBase, Sybase, DB2.

Operating Systems: Windows, UNIX, Linux, Solaris

Theoretical Knowledge: Flume, Solr, Micro Strategy, Beam

PROFESSIONAL EXPERIENCE

Confidential, Richmond VA

Cloud Engineer / Architect

Responsibilities:

• Design, develop and ingest data from different sources in Redshift and Snowflake from S3 buckets
• Transformations using spark Scala and ingest in different stages of the project
• Validation and testing of the data quality and supporting QA.
• Written common framework to support any kind of file reading and ingest to Postgres DB and Redshift
• Building framework for data pipeline from end-to-end.
• Developing POC for various applications.

Environment: AWS, Arow, Spark 2.4, Postgres, Github, Python, Shell Scripting, python, Redshift, Snowflake, Airflow.

Confidential, Irving. TX

Big Data Consultant

Responsibilities:

• Design and ingest data from different file formats by reading from S3 buckets
• Transformations using spark Scala and ingest in different stages of the project
• Validation and testing of the data quality and supporting QA.
• Written common framework to support any kind of file reading and ingest to Postgres DB
• Written parallelized writing and reading of data from Postgres DB.
• Production support and automation of jobs in different stages.

Environment: AWS, Nifi, Spark 2.4, Postgres, GitLab, Python, Shell Scripting, Java, python, Elastic Search.

Confidential, New Jersey, NJ

Big Data Architect / Developer.

Responsibilities:

• Design and ingest data from Greenplum and Oracle.
• Estimation of infrastructure and capacity planning of the cluster.
• Installation and maintenance of the cluster along with admin team.
• Giving permissions and ensuring LDAP and Kerberos authentication.
• Performance tuning and running jobs in Hadoop cluster.
• Hadoop cluster monitoring whether it is up and running all the time and health check.
• Clearing the logs and backup for disaster recovery.
• Interacting with various teams to implement best practices and contribute to POC’s on different use cases.
• Transformations in Scala and python using Spark framework.
• Writing ingestion scripts in through Sqoop jobs.
• Automation of data flow, deploying and scheduling jobs in Control-M.
• Benchmarking and performance tuning of impala queries and concurrency tests.
• Involved in production support and coordinating with admin teams to resolve issues.
• Understanding the domain and existing warehouse and migrating the data into Hadoop.

Environment: Cloudera 9.1, Impala, Spark SQL, Greenplum, Oracle, Sqoop, HBase, Kafka, Control +M, SVN, Bitbucket, Python, shell Scripting, Scala, python and Tableau.

Confidential, TX

Big Data Architect

Responsibilities:

• Architecting and designing the pipeline of the data.
• Gathering the requirements from SME.
• Writing ingestion scripts in through jobs.
• Creating pipeline for Tibco streaming data and store in HBase.
• Understanding the domain and existing warehouse and migrating the data into Hadoop.
• Assigning tasks to offshore team and monitor progress through agile methodology.

Environment: Horton Works 2.3, Hive, Spark streaming, Spark SQL, Teradata, DB2, shell scripting, Sqoop, HBase, Control +M

Confidential, Worcester, MA

Big Data Architect

Responsibilities:

• Determining the cluster size based on data size and incremental data.
• Installation and maintaining HDP 2.3 on 4 node cluster.
• Involved in preparing technical design document for
• Architecting and designing the end - to - end pipeline of the data.
• Responsible for assigning and implementation of development tasks.
• Writing UDTF in Java to parse the Acord XML and store in Hive.
• Implementing workflows and automation though Nifi.

Environment: Horton works 2.3, Spark, Java, Hive, Nifi, Sqoop, Oracle, DB2, Power BI, and Control-M.

Confidential

Hadoop Engineer

Responsibilities:

• Architecting and designing the pipeline of the data.
• Gathering the requirements from business.
• Interacting with data modelers and creating the refined data scripts.
• Curation through Pig and scrubbing of data and loading to the cloud environment.
• Creating Hive External tables to Data scientists for analytics.
• Involved in optimization of data and performance tuning of hive queries.
• Supporting visualization team to create dash boards on QlikView.

Environment: Horton works 2.3, Spark, Java, Hive, Nifi, Sqoop, Oracle, DB2, Power BI, and Control-M, Microsoft Azure HDInsights on Windows and Linux, Spark, Python, Hive, BLOB Storage, ETL (Informatica), QlikView, PowerShell, Pig.

Confidential, New York

Developer /Architect

Responsibilities:

• Architecting and designing the project from start.
• Interacting with data scientists and gathering requirements.
• Installation of 4 node HDP cluster and creating user profiles.
• Developing and integrating Scala application on Spark (RDD’s) using Hadoop Cluster.
• Coding and implementation of Hive tables using monthly partitions.
• Importing data from Oracle using Sqoop and other files through FTP.
• Developing and scripting in Python and Scala.
• Developing and running scripts on Linux production environment.
• Reporting and visualizations using Business objects. (Tableau)

Environment: Cloudera 5.3, RHEL, R, Sqoop, SparkR, Oozie, Hive, Scala, python, BO.

Confidential

Developer /Architect

Responsibilities:

• Analyze Barclays enterprise architecture
• Solution proposal for migrating from Teradata to Hadoop platform
• Design the actual solution
• Analyze tools and technologies required and support provided by existing software’s latest versions and its support for Hadoop HDFS.
• Propose for technological upgrades
• Hadoop cluster sizing
• Develop a solution for end-to-end data flow from Oracle to HDFS, access data available in HDFS from Informatica, push/pull of data available in Teradata into and from HDFS

Environment: Java 1.7, Cloudera 4, Sqoop 1.4.3, Hive 0.96, RHEL, Eclipse Helios, Informatica 9.5, Teradata, Oozie, Zookeeper, Oracle.

Confidential

Developer /Architect

Responsibilities:

• Responsible for performing in-depth analysis and conceptualization of Retail Banking Customer 360-degree view.
• Responsible for creating use cases/functional requirements.
• Responsible for designing the user interface.
• Responsible for creating entity model design and user interface creation using AppBuilder.
• Responsible for the overall solution delivery developed using InfoSphere Data Explorer.
• Created the estimation and work breakdown structure for the solution.
• Planned the development and helped the team resolve technical issues.
• Worked with Data Explorer product development team to resolve technical issues and identify solutions.
• Published solution offering document for this solution.
• Responsible to take business requirements to process credit card historical data from Banking SME.
• Responsible to perform loading of credit card historical data into Hadoop.
• Responsible to design and develop Map/Reduce programs for analytics purpose to process the credit card historical data and generate output which is further indexed using BigIndex API for visualization purposes.
• Involved in setting up 50 node Hadoop cluster for executing the solution

Environment: InfoSphere Data Explorer 8.2.3, Hortonworks, DB2 9.7, Linux, Hive, HBase, Zookeeper.

Confidential

Project Lead

Responsibilities:

• Developed classes for end-to-end framework for interacting with Hadoop.
• Worked on Hive Queries to accomplish insert and update on data through joins and functions including UDFs.
• Participated extensively in the design and development of the project.

Environment: Apache Hadoop, Hive, Java.

Confidential

Developer / Project Lead

Responsibilities:

• De-duplicate incoming data using MapReduce written in Java
• Write transformation queries in Hive
• Write UDF in Hive

Environment: Hadoop, Hive, Amazon Web Services, Java.

Confidential, Dallas, TX

Project Lead

Responsibilities:

• Managing and leading the project team
• Detailed project planning and controlling
• Managing project deliverables in line with the project plan
• Coach, mentor and lead personnel within a technical team environment
• Recording and managing project issues and escalating where necessary
• Monitoring project progress and performance
• Providing status reports to the project sponsor
• Managing project training within the defined budget