SUMMARY:

• Experienced IT professional seeking a leadership position driving strategy, management, planning, development, team building and end - to-end solution with focus on leveraging information technology, Big Data and data science to maximize business values.
• Developer / DevOp with over 8 years of development, administration and programming experience with primary focus in "Big Data" and emphasis on Hadoop
• Spark RDD and Spark DF development and DevOp using Python and Scala
• Worked with Hadoop, HBase, Cascading, Zookeeper, Oozie, Hive, HiveQL, MapR, MogoDB, Pentaho & Pig
• DevOp with AWS Cloud, EC2, EMR, RedShift, S3 etc.
• Experienced with using Docker CLI on multiple projects
• Experienced with using Rancher and RancherOS
• Experienced with using GlusterFS and CEPH
• Supported Hadoop and Verita as administrator about 7 to 700 nodes.
• Supported and administrated users accounts and security rules with BigData domain.
• Worked with Application Servers, Tomcat, Oracle and MySQL.
• Experienced with distributed systems, large scale non-relational data stores, map-reduce systems, data modeling, database performance, and multi-terabyte data warehouses.
• Experienced in a SaaS environment that has an agile development process.
• Experienced in Java, Python or other object oriented programming language.
• Experienced in Mobile, Linux, Unix, Android, MAC platform.
• Experienced with server management, server operating systems(Windows, Linux, Unix), and VMware.
• Experienced in testing APIs, Restful APIs.
• Experienced with full SW Lifecycle and Agile practice .
• Experienced with scripting language, in Perl, Python, Ruby, QTP for automated and performance tests.
• Experienced with testing DNA sequence machines SW for regulatory approval.
• Experienced testing networking and storage technologies, protocols and hardware.
• Experienced with web services and SOAP UI testing.
• Experienced with Oracle, TOAD, and SQL.
• Experienced in testing UI technologies such as HTML5.
• Experienced in Big Data/ Hadoop / Cassandra/Memcached/NoSQL/MapReduce.
• Lead and coached about 2 to 20 engineers onshore, offshore and nearshore.
• Lead the hiring process.
• Big/data and analytics platform administrator (Vertica, Greenplum, Hadoop, Hive).
• Experienced with Display Advertising, Behavioral Ad Networks .
• Recommended Engines, Personalization.
• Experienced with Data Analytics, Data mining, Predictive Modeling.

WORK EXPERIENCE:

Confidential, Long Beach, CA

Big Data Developer

Responsibilities:

• Working on processing big volumes of data using different big data analytic tools including Spark Hive, SQOOP, Pig, Flume, Apache Kafka, OOZIE, HBase, Scala.
• Coded the real-time with Spark Streaming and Apache Confidential to Store the data in Hive and HBase.
• Worked as a Hadoop consultant on (Map Reduce/Pig/HIVE/SQOOP)
• Loaded and transformed large sets of structured, semi structured and unstructured data using Hadoop/Big Data concepts.
• Experience in AWS, implementing solutions using services like (EC2, S3,)
• Worked using Apache Hadoop ecosystem components like HDFS, Hive, SQOOP, Pig, and Map Reduce.
• Utilized NOSQL database HBase for loading HIVE tables into HBase tables through Hive-HBase integration which was consumed by Data scientist team.
• Worked on setting up and configuring AWS's EMR Clusters and Used Amazon IAM to grant fine-grained access to AWS resources to users.
• Implement enterprise grade platform (mark logic) for ETL from mainframe to NOSQL (cassandra)
• Working with Struts MVC objects like Action Servlet, Controllers, validators, Web Application Context, Handler Mapping, Message Resource Bundles, and Form Controller.
• Created own shiny-server on Linux Centos OS and deployed reports on server.
• Extensively development experience in different IDE like Eclipse, Net Beans and IntelliJ.
• Developed REST API for test server for Express Router middleware based API's with Mongo DB integration. Strong Experience in developing user interfaces with HTML, DHTML, JSTL, XSD, XML and CSS. Worked with scripting languages like JavaScript, AJAX and JQuery.
• Exposed to Agile environment and familiar with tools like JIRA, Confluence.
• Sound knowledge in Agile methodology- SCRUM, Rational Tools.
• Working in querying data using Spark SQL on top of Spark engine.
• Involved in complete SDLC of project includes requirements gathering, design documents, development, testing and production environments. Packaged Spark development environment into custom vagrant box.
• Lead architecture and design of data processing, warehousing and analytics initiatives.
• Used Apache Confidential for ingestion of data from the IBM MQ's (Messages Queue)
• Identify query duplication, complexity and dependency to minimize migration efforts Technology stack: Oracle, Hortonworks HDP cluster, Attunity Visibility, Cloudera Navigator Optimizer, AWS Cloud and Dynamo DB.
• Developed custom code to read the messages of the IBM MQ and to dump them onto the NifiQueues.
• Creation of a User Interface to search and/or view content within the cluster by using solar cloud.
• Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement.
• Developed dynamic JSP pages with Struts. Used built-in/custom Interceptors and Validators of Struts.
• Enable and configure Hadoop services such as HDFS, YARN, Hive, Ranger, Hbase, Kafka, Sqoop, Zeppeline Notebook and Spark/Spark2.
• Developed different kinds of interactive graphs in ER studios .

Environment: JDBC, NOSQL, Mongo DB, Alteryx, Spark, YARN, HIVE, Pig, Scala, Confidential, intellij, AWS EMR, Python, Hadoop, MYSQL.

Confidential, Santa Clara, CA

Sr. AWS Cloud DevOp

Responsibilities:

• DB development and DB performance tuning
• Managing and confitguring AWS services including EC2, S3, SQS, RedShift, ElastiCache, Cloudwatch, etc.
• Docker Master and install/deployment
• Worked on DB persistence with Java Hibernate development in AWS
• Redshift performance tuning and testing
• Spark RDD and Airflow workflow designing and testing
• Report design using MicroStrategy
• Build lead recommender systems for Confidential website
• Scala functional programming with Spark streaming
• Solr Programming
• Hortonworks Programming.
• VM programming .
• Docker programming
• Using Docker CLI on multiple projects.

Confidential, San Francisco, CA

Sr. Big Data DevOp

Responsibilities:

• Provision, configure and maintain cloud resources including server instances, load balancers, security groups, DNS management, certificates in AWS and company data center
• Manage and maintain development and production systems. Troubleshoot issues as they arise by viewing logs, attempting to reproduce in a controlled environment. Apply periodic updates, patches, etc to servers.
• Ensure system uptime by monitoring production systems and responding to issues with corrective action.
• Ensure network uptime and troubleshoot any issues or outages to ensure business continuity
• Diagnose and troubleshoot day to day issues users may experience with their PC and Mac computers - network connectivity, printing, etc.
• Spark RDD and Spark DF development.
• Docker Master and install/deployment.
• Solr programming
• NoSQL database design and Spark SQL development.
• Unix/Hadoop administrator and development
• Using Docker CLI
• Expertise with Splunk UI/GUI development and operations roles.
• Prepared, arranged and tested Splunk search strings and operational strings.
• Managed the Splunk licenses and restricted the daily indexing limit.
• AWS EMR / Redshift administrator/development
• Cloudera Administrator/development/DevOps
• Cloudera Navigator Administrator/development/DevOps
• Kerbero Administrator.
• Zoomdata administrator.
• Using GlusterFS and CEPH.
• Big data platform design and architecture/development
• Hive and Hbase development.
• Security office for big data
• POC all other big data tools.
• Working with RancherOS
• Work with scripts for Oozie and Hue systems.
• Cassandra, MongoDB, Hbase, Hive development.
• Cloudera Hue and Solr development.
• Data governance policy/management and big data security.
• Big data security and Kerberos development.
• Multi-tenant set up and development.

Confidential, San Francisco, CA

Sr. DataBase Architect

Responsibilities:

• Converting large SAS SQL scripts to HiveQL for Data Science
• Converting Oracle scripts to HiveQL programming for Data Science
• Architecture No-SQL databases and coding the MapReduce functions for over 10 years’ data about emails campaigning results analytics
• Working with data science to prototype data model and programming the business logic with Hive, Pig
• Solr programming
• Building database application on Hive, Pig, Hbase, MongoDB, Canssdra, Spark, Solr and shark on a large hadoop nodes cluster in house
• Backend No-SQL database design, development, architecture and testing
• Data gaverment

Confidential, Redwood City, CA

Backend Big Data/Hadoop Engineer

Responsibilities:

• Spark RDD and Spark SQL/Hive SQL.
• No-SQL database modeling.
• AWS cloud computing and architecture with Hadoop on Big Data.
• Build cloud application on AWS, S3, EMR, Hive, Pig, Hbase, MongoDB, Canssdra, Spark.
• Backend No-SQL database design, development, architecture and testing.
• Solr programming

Confidential, San Jose, CA

Big/Data and BI/BW Consultant

Responsibilities:

• Create and maintain scripts and programs to Extract, Transform and Load data
• Automate ETL and aggregation processes using BASH/PHP and/or other scripting languages
• Create logging and monitoring elements to all Data Warehouse and ETL processes to allow Operations team to monitor
• Respond to Data Warehouse and ETL process alerts, with support from operations team.
• Create, maintain and automate processes to distribute data warehouse extracts to various users
• Expertise with Splunk UI/GUI development and operations roles.
• Continually monitor measure and improve all data warehouse and ETL processes for speed, reliability and accuracy.
• Perform analyses and execute improvements regarding overall data quality, including making recommendations to the business regarding inputs.
• Source and/or create tools to deliver monitoring metrics and dashboard reports to allow the rest of the company to understand the quality and timeliness of data warehouse data.
• Install and integrate 3rd party data maintenance tools such as address cleaning software, etc.
• Use new technologies to service data warehousing needs such as hadoop, column databases, etc.
• Cisco big data analytics platform development, testing and deployment
• Working on data integration application design, development, and testing
• Using Pentaho data integration tool to create data integration jobs
• Using MapR to run Hadoop / YARN jobs
• Moving enterprise data from Oracle to Hive for big data platform
• Moving Confidential, voice, phone, video and email enterprise data to Hive and build data mart.
• Writing use story, task, plan and testing cases
• Writing scripts for automating testing
• Writing design document for enterprise data on board
• Analysis deployment log for errors
• Monitoring big data staging environment
• Monitoring hive staging environment

Confidential, Santa Clara, CA

Big Data DevOps consultant

Responsibilities:

• Building and maintaining production systems on AWS using EC2, RDS, S3, ELB, Cloud Formation, etc. and familiarity interacting with the AWS APIs. You should be equally comfortable in a traditional datacenter setting.
• Using script languages (Python and/or Ruby) as well as script environments like bash
• Administering Linux (Centos, RHEL, Ubuntu) systems.
• Using Puppet, Chef, Ansible, or Salt.
• Monitoring, metrics, and visualization tools for network, server, and application status (Zenoss, Sensu, Nagios, Graphite, Collectd, Ganglia, etc.)
• Using IPS, WAF, and additional security layers (LDAP, SSO, 2Factor)
• Maintaining RDBMS (PostgreSQL and MySQL). Bonus points for NoSQL (Cassandra, DynamoDB, Couchbase, Mongo)
• Spark RDD development and No-SQL database modeling
• Design/Architecture Big Data Hadoop Testing Frameworks
• Building Hadoop cluster in Cloud and local
• Working with AWS cloud environment: EMR, Redshift, S3
• Working with Scala, Java, Python, Kafka, akka, SQL, MongoDB, JSON, Avro, Tableau
• Working with Git, SBT, Ant, Maven, Ganglia, Jenkins,
• Working with Hadoop, Hbase, Zookeeper, Oozie, Scalding Spark Shark
• Testing automation and coding in Scala
• Worked with ETL and reporting tools (OBIEE, SAP etc.)

Confidential, Milpitas, CA

Big/Data and BI consultant

Responsibilities:

• Hadoop /MapR BW/BI data warehouse project.
• Big data move from Informatica and TD.
• Load data from different source to new build Hadoop analyst platform
• Build Hadoop QA team
• Using Hadoop /Hive/Sqoop/bash to deployment the data load and query data.
• Monitor QA environment for Hadoop problems
• Testing report form MapR report tool
• Provide the ideas/working process to other team
• Review other reports and code to understand coding logic
• Managed a technical team or functioning as a team lead.
• Worked with Hadoop stack (e.g. MapReduce, Sqoop, Pig, Hive, Hbase, Flume). related/complementary open source software platforms and languages (e.g. Java, Linux, Apache,
• Perl/Python/PHP, Chef).
• Worked with ETL (Extract-Transform-Load) tools (e.g. Informatica, Talend, Pentaho).
• Worked with BI tools and reporting software (e.g. Microstrategy, Cognos, OBIEE, Pentaho).
• Worked with analytical tools, languages, or libraries (e.g. SAS, SPSS, R, Mahout).
• Supported business development activities to shape and communicate proposed solutions to client executives - Implemented of ETL applications:
• Implemented of reporting applications
• Application/implementation of custom analytics support
• Administrated of relational databases
• Data migration from existing data stores
• Infrastructure and storage design
• Developed capacity plans for new and existing systems.

Confidential, San Jose, CA

Big/Data and BI consultant

Responsibilities:

• Conducted detailed design applications developed on Hadoop platforms (Feature testing, Regression Testing, Acceptance Testing and Sanity Testing)
• Implemented Business analysis tools with Hadoop MapReduce scripts from ETL data to data warehouse for BI and enterprise analysis platform
• Administrated Log analysis scripts for Business Analyst tool with HDFS Hadoop (file system level)
• Advised the Hadoop and analytical workloads IO optimizing solutions
• Running benchmarked Hadoop/HBase clusters
• Supported development of Hadoop and Vertica Analytics Platform activities.
• Tested NetApp Open Solution for Hadoop(NOSH) settings
• Supported and scaled Hadoop systems
• Supported cluster failed over test and document the results with various configurations
• Administrated enterprise-grade storage arrays and eliminated Hadoop network bottlenecks
• Supported hot-pluggable disk shelves, added storage and administrated services
• Supported NFS and HDFS file systems
• Loaded network-free hardware RAID
• Day-to-day support Hadoop hardware and software issues
• Used automation scripts to run performance testing of the new storage OS design.

Confidential, Concord, MA

Big Data System Consultant

Responsibilities:

• Supported about over 100 projects and many functional teams
• Supported analytic massive amounts of data and BI
• Communicated and tracked defects to closure
• Administrated with large data set testing with BigData NoSQL Hadoop and Oracle, DB2 scripts
• Wrote and reviewed scripts and executed scripts, tracked defects
• Expertise with Splunk UI/GUI development and operations roles.
• Used scripts languages in Perl and Python for Automations testing and test data condition
• Used scripts languages for ETL load with BW.

Confidential, Walnut Creek, CA

Middleware Architect

Responsibilities:

• Designed all middleware with 3000 interfaces for BigData.
• IBM MQ design/ architecture support.
• Designed QA environment and production support.
• Designed Message broker and production support.