SUMMARY

• 7+ years of experience in Analysis, Design, Development, Management and Implementation of various stand - alone, client-server enterprise applications, refining and scaling data management and analytics, procedures, workflows and best practices.
• Experienced in extracting Real time feed using Spark Streaming and convert it to RDD and process data in the form of Data Frame and save the data as Parquet format in HDFS.
• Experienced in handling large datasets using partitions, Spark in Memory capabilities, Broadcasts in Spark, effective & efficient joins, transformation and other during ingestion process itself.
• Worked on migrating legacy SAS programs into Spark transformations using Spark and Python.
• Expertise in database platforms and as well worked on Hadoop, Spark, AWS EC2, AWS S3 and monitoring resources for better understanding of functioning of systems.
• Hands on experience in loading data from UNIX file system to HDFS.
• Good understanding of Object-Oriented Technologies and Relational Database Systems.
• Good Knowledge of Agile Methodologies (Scrum).
• Strong experience in Shell Scripting, SQL Server, Linux, and Open stack.
• Experienced in writing SQL Queries, Stored procedures, functions, packages, tables, views, triggers.
• Building a Data Quality framework, which consists of a common set of model components and patterns that can be extended to implement complex process controls and data quality measurements using Hadoop.
• Strong Experience in working with Python ORM Libraries including Django ORM, SQLAlchemy.
• Experienced of software development in Python (libraries used: libraries- Beautiful Soup, NumPy, SciPy, matplotlib, python-twitter, Pandas data frame, network, urllib2, MySQL dB for database connectivity) and IDEs - sublime text, Spyder, PyCharm.
• Experienced in developing web-based applications using Python, Django, PHP, XML, CSS, HTML, DHTML, JavaScript and jQuery, RUBY, AJAX.
• Expertise in working with GUI frameworks - Pyjamas, Python.
• Good knowledge of web services with protocols SOAP, REST.
• Have Experience in working with server-side technologies including databases, restful API and MVC design patterns.
• Experienced in NoSQL technologies like MongoDB, Cassandra and relational databases like Oracle, SQLite, PostgreSQL and MySQL databases.
• Very strong experience writing API's and Web Services in PHP and Python.
• Superior Troubleshooting and Technical support abilities with Migrations, Network connectivity, and Security and Database applications.
• Expert level skills in HTML, CSS, and JavaScript including familiarity with common libraries like jQuery, Foundation, Bootstrap and Backbone.
• Skilled in debugging/troubleshooting issues in complex applications.
• Experience in working with different operating systems WINDOWS, LINUX, and iOS.
• Expert in maintaining technical documentation for projects.
• Good analytical and problem-solving skills and ability to work on own besides being a valuable and contributing team player.
• Excellent Interpersonal and communication skills, efficient time management and organization skills, ability to handle multiple tasks and work well in a team environment.
• Hands-on experiences in writing and reviewing requirements, architecture documents, test plans, design documents, quality analysis and audits.

TECHNICAL SKILLS

Languages: Python, C, Ruby, shell scripting.

Web Design: HTML5, XHTML, CSS3, JSP, AJAX

Databases: Microsoft SQL Server, SQLite, MySQL, PostgreSQL, DB2, MongoDB, Cassandra, Redis

Frameworks: Django, Flask, Pyramid, Pyjamas, Jython, Angular JS, Node JS, Spring, Hibernate

Python Libraries: Report Lab, NumPy, SciPy, Matplotlib, HTTPLib2, Urllib2, Beautiful Soup, Pickle, Pandas

Application and Web Servers: Apache Tomcat, JBOSS, WE Brick, Phusion Passenger

Bigdata Ecosystems: HDFS, Apache Spark, AWS EMR, PySpark

Version Control Systems: CVS, SVN, Git and GitHub.

Deployment tools: Amazon EC2, He Roku

Operating Systems: Windows, Linux, Unix

Protocols: HTTP/HTTPS, TCP/IP, SOAP, SMTP

Other Tools: MS Office (MS-Excel, MS-PowerPoint, MS-Project 2013), Visio 2013

PROFESSIONAL EXPERIENCE

Confidential, Houston, TX

Sr. Python/Data Engineer

Responsibilities:

• Analysed the SQL and SAS scripts and designed the solution to implement using Pandas and PySpark.
• Built web pages using HTML, CSS, JavaScript and jQuery for QC Reports on data.
• Developed a python package to connect to Teradata from Spark and generating QC Reports.
• Worked on high configuration EMR cluster to run Spark jobs.
• Developed a web application using Python and Django.
• Worked in Agile methodology attending the daily stand up and completing tasks in sprints.
• Analysed and resolved data load issues by working with business and technical teams.
• Managed to build new connections to databases like ORE, Snowflake, Redshift and Teradata from Spark and Python.
• Rewrote the legacy Teradata SQL queries to Amazon Snowflake as part of data base migration.
• Analyse requirements at the business meetings and strategize the impact of requirements on different applications.
• Analyse, design and migrate systems using latest Technologies like python, spark and cloud infrastructure using Amazon Web Services in order to provide long-term supportability and sustainability.
• Based on the new or updated business requirements, design and implement the Rules for Processing Workflows using latest python and Spark frame work versions.
• Involved in Business requirements, Data analysis and System design meeting.
• Created entire web application using Python, Django and MySQL.
• Used HTML, CSS and JavaScript to create front end pages using Django Templates and wrote Django Views to implement application functions and business logic.
• Extracted data from multiple sources, integrated data into a common data model, and integrated data into a target database, application, or file using efficient programming processes.
• Designed and developed data management system using MySQL and optimized the database queries to improve the performance.
• Added support for Amazon AWS S3 and RDS to host static/media files and the database into Amazon Cloud .
• Tuned the code with performance and consistency as the main factors of consideration.
• Developed entire frontend and backend modules using Python on Django Web Framework.
• Designed and developed data management system using MySQL .
• Wrote Python scripts to parse XML documents and load the data in database.
• Using GitHub version control tool to coordinate team-development.
• Responsible for debugging and troubleshooting the web application.

Environment: Python 2.7, SQL, Spark 2.1.0, Snowflake, Amazon S3, Elastic Map Reduce, Django 1.9, Java Script, HTML, XHTML, jQuery, JSON, XML, CSS, MySQL, Bootstrap, Git, Linux.

Confidential, Chicago, IL

Sr. Python/Django developer

Responsibilities:

• Participated in all the stages of software development lifecycle like design, development, and implementation and testing.
• Django Framework that was used in developing web applications to implement the model view control architecture.
• For database access Django API's have been used.
• Business logic implementation, data exchange, XML processing and graphics creation has been done using Python and Django.
• Views and Templates were developed with Python and to create a user-friendly website interface Django's view controller and template language is used.
• Developed UI using CSS, HTML, JavaScript, AngularJS, jQuery and JSON.
• DB2 SQL Procedures and UNIX Shell Scripts were designed and developed for Data Import/Export and Conversions.
• A Django dashboard with custom look and feel for end user has been created after a careful study of the Django admin site and dashboard.
• Unit Test Python library was used for testing many programs on python and other codes.
• JIRA was used to build an environment for development.
• Different testing methodologies like unit testing, Integration testing, web application testing was performed.
• For the extraction of data from the database various queries were written and a database was created using MYSQL.
• Search engine optimization by replacing existing database with MongoDB (NoSQL Database).
• To build and populate DB and to ensure the standards the team of QA was also collaborated.
• In order to avoid the reloading of the entire web page to update small portions, AJAX in UI was used.
• For sending and receiving the data between multiple systems RESTful Web-Services were implemented.
• Features for dashboard were developed and tested using CSS, JavaScript, Django, and Bootstrap.
• Git repository was created and added to GitHub project.
• An application was developed in Linux environment and dealt with all commands.
• Performance and consistency being the main factors of consideration the code was tuned.
• Actively worked as a part of team with managers and other staff to meet the goals of the project in the stipulated time.

Environment: Python 2.7, Django 1.6, Java Script, HTML, XHTML, Angular JS, jQuery, JSON, XML, CSS, MySQL, Bootstrap, Git, Linux, Pharms, requests.

Confidential, Seattle, WA

Python/Django developer

Responsibilities:

• Used Django framework for application development.
• Booting up nodes using prebuilt images on Amazon EC2 .
• Bootstrapping Amazon EC2 nodes with software you want running when your nodes boot up.
• Uploading, copying, downloading, and deleting files using Amazon S3 .
• Assisted in reduction of cost and optimization of supplier selection for the CRM Applications.
• Used several python libraries like Python, Numbly and Matplotlib .
• Design, develop, test, deploy and maintain the website.
• Designed and developed the UI of the website using HTML, XHTML, AJAX, CSS and JavaScript .
• Developed entire frontend and backend modules using Python on Django Web Framework .
• Wrote python scripts to parse XML documents and load the data in database.
• Generated property list for every application dynamically using python.
• Using Subversion version control tool to coordinate team-development.
• Responsible for debugging and troubleshooting the web application.
• Build all database mapping classes using Django models and Cassandra .
• Used Pandas API to put the data as time series and tabular format for east timestamp data manipulation and retrieval.
• Designed and developed data management system using MySQL .
• Built various graphs for business decision-making using Python matplotlib library.
• Created Python and Bash tools to increase efficiency of call centre application system and operations; data conversion scripts, AMQP/RabbitMQ, REST, JSON, and CRUD scripts for API Integration.
• Created server monitoring daemon with Psutil , supported by Django app for analytics which I created. Also researched big data solutions with Cassandra database.
• Fetched twitter feeds for certain important keyword using python -twitter library.
• Experienced in Agile Methodologies and SCRUM Process.
• Worked in development of applications especially in UNIX environment and familiar with all of its commands.
• Collaborated with internal teams to convert end user feedback into meaningful and improved solutions.
• Resolved ongoing problems and accurately documented progress of a project.

Environment: Python 2.7, Django 1.4, SciPy, Pandas, Bugzilla, SVN, C++, Java, jQuery, MySQL, Linux, Eclipse, Shell Scripting, HTML5/CSS. Red hat Linux, Apache, Cassandra