We provide IT Staff Augmentation Services!

Sr. Data Engineer Resume

Seattle, WA

SUMMARY

  • A collaborative engineering professional with substantial experience designing and executing solutions for complex business problems involving large scale data warehousing, real - time analytics and reporting solutions. Known for using the right tools when and where they make sense and creating an intuitive architecture that helps organizations effectively analyze and process terabytes of structured and unstructured data.
  • SDLC: Over 16+ years of experience in planning andRequirement Analysis, Defining Requirements, Designing the product architecture, Building or Developing the Product, Testing the Product, Deployment and Maintenance.
  • Managingthe everyday business operations associated with BI, such as analytics, reporting, information delivery, data preparations, data exploration, and data warehousing. Technical expertise and general business skills (communication, presentation, leadership, analytical thinking)
  • 3 years of experience and certification in Big Data projects, collecting, storing, processing, and analyzing of huge sets of data, choosing optimal solutions in Spark and Hadoop ecosystem, then maintaining, implementing, and monitoring them.
  • Provided competitive intelligence and actionable insight to financial, operational and sales teams as well as C-level management by identifying drivers corporate performance. Supervised a team of 3 analysts and developers, managed vendors and administered development budget.
  • Ability to write and maintain technical documentation to describe Reports development, logic, coding, testing, changes, and corrections.Utilized Best Practices and Proven Methodology in BI development process.
  • Experience and training in Confidential Web Services EMR, S3, EC2, IAM, Route53, Databases(RDS, DynamoDB, Redshift), VPC, Lambda, EBS, EFS, Glue, Athena, SQS, SNS, API Gateway, Kinesis.
  • Business Intelligence: Over 8+ years as an application developer with expertise in implementing and developing Data Warehousing/Business Intelligence solutions applying advance techniques from Kimball Group, using IBM BICognos and OBIEE, in-depth and comprehensive experience in design, development, testing, Security and Support for Data warehousing and Client/Server projects
  • Cognos BI: Development and implementation of Cognos10.x, Cognos 8.x, Report Studio, Query Studio, Analysis Studio, Transformer, Power play,Cognos 8
  • Installed and configured Cognos 8.2/8.3/10 BI, BI modeling, Report Studio, Query Studio and Analysis Studio. Cognos8.3, Cognos 8.2, Report Net, Query Studio, Report Studio, Transformer, Power play, scheduled the Reports based on the database events using Triggers.
  • Expert in developing and designing architectural solutions according to project/business need
  • Oracle BI: Experience in development and implementation of OBIEE suite: BI Admin, BI Answers (Interactive Dashboards, Scorecard and Strategy Management, Spatial Visualizations and Analytics), BI Publisher, BI Server
  • Designed and developed dynamic dashboards to evaluate the Company’s performance. Implemented Row Level Security, Object Level Security, Package Security, Folder Security and created user classes as per the requirement. Created complex Lists, Cross Tabs, Charts and Repeater in Cognos 8.x/10.x and OBIEE.
  • Created JavaScript solutions in Cognos reports to enhance the out-of-box capabilities of the tool.
  • Developed logical models providing Business Relations implementing Star schema and Snowflake schema concepts.
  • Expertise in developing Database Schemas like Star Schema and Snowflake Schema used in relational, dimensional and multidimensional modeling.
  • Identified the Facts and Dimensions using Erwin Data modeling tool to represent the Star Schema Data Marts.
  • Hands on experience with OLAP tools such as Transformer in creating multi-dimensional cubes and Power Play for Windows.
  • Complex Report building with Report Studio to customize the functionality in prompt page, Master detail Relationships, Drill through in Report page, Customized Prompts and Conditional Formatting.
  • Experienced in Full Life Cycle and Methodology for implementing Data warehouse and Business Intelligence Reporting Systems
  • Experience with SQL in testing and validating reports.
  • Expertise in creating ETL processes in Microsoft Integration Services 2003/2008 and Informatica
  • Expertise in Building Cube, Dimensions using TM1 Turbo Integrator processes and cognos planning by Applying rules to the cube and worked on improving performance of the cube.
  • Expertise in analytical and problem solving skills. Identify the root cause for the problem and deliver the solution with in stipulated time.
  • Manage Server with Windows Server 2003/2008, backing up and restoring data, changing group memberships, checking event logs, creating administrative scripts, creating user and group accounts, deploying and upgrading software, installing a DHCP server, managing applications on a local computer, managing applications remotely, management tasks for disks and volumes, file and folder management, managing network printers, managing servers remotely, managing services, monitoring network traffic, monitoring security-related events, monitoring server performance, resetting user passwords, safeguarding system, scheduling tasks, setting up DNS, setting up TCP/IP, setting user and group security, installing Active Directory Server, installing Apache Tomcat.
  • Created complex queries in Microsoft SQL Server 2005 (PL/SQL) and TOAD to discover various data sources.
  • Used DevOps mode to code, build, test, package, release, configure and monitor aspects of the development and delivery process.
  • Installed, configured and managed Oracle DB 7/8/9i/10g/12g in Windows Server 2003/2008 and Linux
  • Possess ability for critical thinking, analysis, good interpersonal and communication skills. Team-oriented, technically motivated and creative. User oriented with a desire to learn client's business requirements.

TECHNICAL SKILLS

Cloud Computing: Confidential Web Services Architect, covering resources like EMR, S3, EC2, IAM, Route53, Databases (RDS, DynamoDB, Redshift), VPC, Lambda, EBS, EFS, Glue, Athena, SQS, SNS, API Gateway, Kinesis.

BI Tools: Cognos (8.3,8.4,10.1,10.2) (Report Studio, Query Studio, Event Studio, Analysis Studio, Cognos Connection, Framework Manager), Power Play Transformer, IBM Cognos TM1 10.1/10.2

OBIEE: BI Admin, BI Answers (Interactive Dashboards, Scorecard and Strategy Management, Spatial Visualizations and Analytics), BI Publisher, BI Server Data Warehouse, Data mart, Informatica, Microsoft Integration Services 2003/2008

Big Data: Horton Works and Cloudera(Hadoop, HDFS, clusters, Yarn, Zookeeper, Sqoop, Python, Scala, Spark, Hive, Impala, Avro, Flume, Kafka)

Platforms & OS: Linux, Windows Server 2003/2008, Win NT, Win95, Win2000/2003/Advanced server, WinXp:

Database: Oracle 7.x/8.x/9i/10g/12g, Microsoft SQL Server 7/2000/2005, MS Access97/2000.

Other Tools: DevOps, TOAD, SQL Programmer, Erwin, Jenkins, Microsoft Project

App/Web Servers: IIS 5.0, WebSphere, Apache Tomcat.

Directory Servers: Active Directory Server

Languages: Python, Scala, SQL, PL/SQL, C, C++, HTML, Regex

Java & J2EE: Java, JSP, Java Script

PROFESSIONAL EXPERIENCE

Sr. Data Engineer

Confidential, Seattle, WA

Responsibilities:

  • Implemented Big Data solutions to process billions of records of financial information using EMR, Hadoop ecosystem and Spark with Scala and Python. Performance tuning.
  • Designed and Implemented AWS solutions to create, maintain, orchestrate and monitor data pipelines to ingest, process and load large amounts of data, following best standards in operation excellence, security, performance efficiency, reliable and cost effective fashion.
  • Developed complex scripts in Redshift to process financial data. DDL and DML, performance tuning and Redshift Cluster Management.
  • Created Data warehouse using best standards for reliability, performance and scalability, to store big amounts of data for reporting and analytical purposes.
  • Used Confidential internal tools like Horizon, DJS, Datanet to create data pipelines.
  • Developed Unix Shell Scripting to perform server operations like file manipulation, program execution, cleanup, logging, etc.
  • Developed OBIEE BI administration metadata (physical, local and presentation layer) and create BI reports and analysis.
  • Built requirements from business users and work with other Data Engineers and software developers.

Environment: AWS(EMR, S3, Redshift, Redshift Spectrum, VPC, Glue, Athena, EC2), Apache Spark (Scala and Pyspark), Hive, Zeppelin, Hue, HDFS, Unix shell scripting, IntelliJ IDE. Confidential internal tools like Horizon, DJS (Data pipeline flow), Datanet (ETL).

Sr. Technology Architect

Confidential

Responsibilities:

  • Created ETL data pipelines with Apache Spark to ingest large amounts of data fromiTunes Payments systems (40 TB for two years) in a snowflake schema.
  • Performed Enrichments and derivations on a driver table to load history data.
  • Performance tuning in Spark workflows to ingest very large data sets
  • Apache Hive tuning (Partitioning, Bucketing, queues, File Format, Compression)
  • Build requirements from Business Users
  • Documentation

Environment: Apache Spark, Scala, Python, Hive, Sqoop, IntelliJ IDE.

Sr. Data Engineer

Confidential, NY

Responsibilities:

  • Created Big Data architectures using Confidential Web Services resources like EMR, Glue, RDS, S3, IAM, EC2, Redshift and Spark Cluster to leverage data processes to integrate information digital advertising (3 TB per month).
  • Created Spark applications in Big Data Clusters like Horton Works and Cloudera, using Scala and Python, Hive, Sqoop, Hadoop, HDFS, Yarn, Zookeeper, Kafka.
  • Performance tuning in Spark workflows to ingest very large data sets
  • Apache Hive tuning (Partitioning, Bucketing, queues, File Format, Compression)
  • Designed and Implemented Databases in RDS and Redshift to support the load of data coming from vendors systems.
  • Created Docker containers to generate solutions for data processing.
  • Used DevOps mode to code, build, test, package, release, configure and monitor aspects of the development and delivery process.
  • Trained as an Confidential Web Services Architect, covering resources likeEMR, S3, EC2, IAM, Route53, Databases (RDS, DynamoDB, Redshift), VPC, Lambda, EBS, EFS, Glue, Athena, SQS, SNS, API Gateway, Kinesis.

Environment: Confidential Web Services (EMR, Glue, RDS, S3, IAM, EC2), Big Data Clusters like Horton Works and Cloudera, using Scala and Python, Hive, Sqoop, Hadoop, Yarn, Zookeeper, Kafka, Docker, DevOps.

Hire Now