Python Data Engineer Resume
SUMMARY
- 4+ years of experience as a Python Developer, proficient coder in multiple languages and environments including Python, C, C++, HTML, JavaScript, and SQL.
- Developed various Python scripts to generate reports, send FIX messages (FIX Simulator
- Good experience in developing web applications and implementing Model View Control (MVC) architectureusing server - side applications likeDjango, Django REST, FlaskandPyramid.
- Extensive experience in developing Front End using HTML/HTML5, XML, DHTML CSS/CSS3, JavaScript, React JS, Angular JS (1.X) JQuery, JSON, Node.js, Ajax, JQUERY Bootstrap.
- Expertise in working with different databases like Microsoft SQL Server,Oracle, MySQL, PostgreSQLand Good knowledge in using NoSQL databases MongoDB and Cassandra.
- Proficient in developing complex SQL queries, Stored Procedures, Functions.
- Excellent working knowledge inUNIX and Linux shell environmentsusing command line utilities.
- Application Stress tester, created and stress tested standalone, web - applications and generated graph reports.
- Experience in analyzing data using Big Data Tools likeHiveQL program. Also Experience in usingHbase, Oozie.
- Extensively used SQL, Numpy, Pandas, Scikit-learn, Spark, Hive for Data Analysis and Model building.
- Extensively worked onHadoop, Hive, Spark, Cassandra to build ETL and Data Processing systems having various data sources, data targets and data formats
- Excellent knowledge of Machine Learning, Mathematical Modeling and Operations Research. Comfortable with R, Python, and Relational databases. Deep understanding and exposure of Big Data Eco - system.
- Strong knowledge of Data Structures and Algorithms, Object Oriented Analysis, machine learning and software design patterns.
- Expertized in Implementing Spark using ScalaandSparkSQLfor faster testing and processing of data and Experience inNoSQLdatabases such asHBaseandCassandra.
- Hands on experience with Spark-Scala programming with good knowledge on Spark Architecture and its In-memory Processing and Experience with Unit testing/ Test driven Development (TDD), Load Testing.
- Hands on experience working inWAMP (Windows, Apache, MYSQL, and Python/PHP)andLAMP (Linux, Apache, MySQL, and Python) Architecture.
- Experience withAgile, Scrum and Waterfall methodologies. Used ticketing systems likeJira.
- Knowledge of testing deployment and deployment tools usingHeroku, Jenkins, pylint.
- Expert at version control systems like Git, GitHub, SVN and CVS. Migrated repos from SVN to GitHub.
- Experience in Test driven development for functional and integration testing using Python PyUnit for unit Testing.
TECHNICAL SKILLS
Programming Languages: Python, R, C, C++, and Java
Python Frameworks and Libraries: Python, Django, Flask, Pandas, MySQLDB, XMLDocx.
Web Technologies: HTML, CSS, JavaScript, Bootstrap, JQuery, AJAX, XML, Angular JS, Node JS
Development/ Deployment Tools: Jenkins, pyLint, Docker, Kubernetes
Databases: Microsoft SQL Server, Oracle, MySQL, MS Access, PostgreSQL
NoSQL databases: Mongo DB, Cassandra, HBase.
Cloud Technologies: AWS, Amazon EC2, S3, Heroku, MS Azure.
Operating Systems: Linux 4.x/5.x/6.x, Ubuntu, Red Hat Linux, Windows server 2008, 2012
Version Controls: CVS, SVN, Git, GitHub.
Testing, Issue Tracking and Debugging Tools: Jira,Gitlab.
Automation Testing: Selenium, PyTest.
Development Methodologies: Agile, SCRUM and Waterfall.
PROFESSIONAL EXPERIENCE
Confidential
Python Data Engineer
Responsibilities:
- Utilized Apache Spark with Python to develop and execute Big Data Analytics and Machine learning applications, executed Machine Learning use cases under Spark ML and Mllib.
- Identified areas of improvement in existing business by unearthing insights by analyzing vast amount of data using machine learning techniques.
- Interpret problems and provides solutions to business problems using data analysis, data mining, optimization tools, and machine learning techniques and statistics.
- Involved in data modeling the tables in Cassandra DB. Familiar with all the internal tools of Cassandra NoSQL.
- Created several tables as a part of data modeling and determined the performance of the table through load testing with Cassandra -stress tool.
- Restoring the backups through sstable loader tool in NoSQL database management system Cassandra
- Led discussions with users to gather business processes requirements and data requirements to develop a variety of Conceptual, Logical and Physical Data Models. Expert in Business Intelligence and Data Visualization tools: Tableau, Microstrategy.
- Creating data pipelines using Apache Spark, a big-data processing and computing framework and Updated and maintained Jenkins for automatic building jobs and deployment.
- Worked on machine learning on large size data using Spark and Map Reduce.
- Let the implementation of new statistical algorithms and operators on Big Data and SQL platforms and utilized optimizations techniques, linear regressions, K-means clustering, Native Bayes and other approaches.
- Developed Spark/Scala, Python for regular expression (regex) project in the Big Data Hive environment with Linux/Windows for big data resources.
- Data sources are extracted, transformed and loaded to generate CSV data files with Python programming and SQL queries.
- Stored and retrieved data from data-warehouses using Amazon Redshift.
- Used Data Warehousing Concepts like Ralph Kimball Methodology, Bill Inmon Methodology, OLAP, OLTP, Star Schema, Snow Flake Schema, Fact Table and Dimension Table.
- Refined time-series data and validated mathematical models using analytical tools like R and SPSS to reduce forecasting errors.
- Familiarity in working with TCP/IP, IPv4, and IPv6 protocols in an environment, which provides multithreading, multi tenancy and high availability support at Network Layer.
- Created Data Quality Scripts using SQL and Hive to validate successful das ta load and quality of the data. Created various types of data visualizations using Python and Tableau.
Environment: Python, Django, Big Data, Map Reduce, Spark, Spark MLLib, Tableau, SQL, Excel, VBA, Matlab, AWS, Cassandra, Oracle, MongoDB, SQL Server 2012, DB2, XML, Tableau.
Confidential
Python Developer
Responsibilities:
- Worked with the Stakeholders, gathered requirements developed High level design Detail design documents.
- Developedwebsiteboth frontend and backend modules using PythonDjangoWeb Framework.
- Designed front end website using HTML, CSS, JavaScript, jQuery, Ajax, Bootstrap.
- Design and develop components using Python. Implemented code in python to retrieve and manipulate data.
- Implemented database access using Django ORM.
- Used MySQL as backend DB and MySQL DB of python as database connector to interact with MySQL server.
- Used Restful APIs to access data from different suppliers.
- Developed Python and shell scripts for automation of the build and release process.
- Support the scripts configuration, testing, execution, deployment and run monitoring and metering.
- Used Python and Django creating graphics, XML processing of documents, data exchange and business logic implementation between servers.
- Used Restful API's to gather network traffic data from Servers.
- Supported Apache Tomcat web server on Linux Platform.
- Developed and executed User Acceptance Testing portion of test plan.
Environment: Python, Django, C++, HTML, CSS, JavaScript, Ajax, XML, Bootstrap, MySQL, Shell Scripting, Unix, Linux, Agile, pylint, Jenkins.
Confidential
Software Developer
Responsibilities:
- UsedAgileMethodology andSCRUMProcess.
- Worked with the Stakeholders, gathered requirements, developed High level design and Detail design documents.
- Developed UI usingHTML, AJAX, JavaScript and jQuery and jQueryUI.
- Re-engineered various modules for implementing changes and creating efficient system.
- Developed rich UI web application usingJavaScriptlibraries likejQueryUI, datagrid, jscolor, and highcharts.
- UsedMySQLas backend database and MySQLdb of python as database connector to interact with MySQL server.
- UsedRestful APIsto access data from different suppliers.
- UsedPython and Djangocreating graphics,XMLprocessing of documents, data exchange and business logic implementation between servers.
- Associated with Restful API's to gather network traffic data from Servers.
- SupportedApache Tomcatweb server on Linux Platform.
- Support the scripts configuration, testing, execution, deployment and run monitoring and metering.
- Developed and executed User Acceptance Testing portion of test plan.
Environment: Python, Django, C++, HTML/XHTML, XML, MySQL, Shell Scripting, Unix, Linux, Windows.