We provide IT Staff Augmentation Services!

Sr Consultant Big Data Engineer Resume

3.00/5 (Submit Your Rating)

Dallas, TX

PROFESSIONAL SUMMARY:

  • 7 years of professional IT experience (Business Analysis, Business Intelligence, Troubleshooting) in with 2 years of Big Data Engineering and Analysis
  • Experience with Cloudera Distribution Platform.
  • Hands on experience on major components in Hadoop Ecosystem like Hadoop, Map Reduce, HDFS, Impala, Spark (SparkCore, SparkSQL, SparkStreaming), Hive, Hbase Sqoop, Oozie and Flume.
  • In depth understanding/knowledge of Hadoop - bases platform and various components such as HDFS, JobTracker, TaskTracker, NameNode, DataNode and MapReduce concepts.
  • Experience in implementing complex ETL transformations in Hadoop platform.
  • Strong Experience in Unix shell scripting to automate file preparation and database loads.
  • Experience in managing and reviewingHadoop Log files.
  • Experience with importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
  • Performance optimization of Sqoop, Hive and Spark.
  • Understanding Hive and Pig core functionality by writing custom UDFs.
  • Experienced with the integration of various data sources and file types (RDBMS, Json, Parquet, Avro, Text, CSV and 3rd Party feeds).
  • Experienced in analyzing data using Scala and Python with HiveQL, RDD, DataFrames, D-streams, Pandas and custom MapReduce programs.
  • Experience with Scala and SQL in application development and deployment.
  • Experienced in configuring flume and kafka to stream data into HDFS.
  • Proficient with NoSQL database platform
  • Experience in SDLC - Agile and Waterfall methodology.
  • Experience in Web Services using XML and HTML.
  • Ability to blend technical expertise with strong Conceptual and Analytical skills to provide quality solutions and result-oriented problem-solving technique and leadership skills.
  • Excellent communication skills; creative, conceptual, research-minded, technically competent and result-oriented with problem solving and leadership skills.
  • Experience creating embedded Power BI reports using MS Azure and visual studio.
  • Experience with Power BI data modelling and DAX programming language.
  • Experience with Data Visualization with SSRS, Power BI Desktop, and Power BI Services.

TECHNICAL SKILLS:

Big Data Ecosystem: Hadoop, Spark, HDFS, Map Reduce, Hive, Impala, HBase, Zookeeper,Sqoop, Kafka and Flume

Methodologies: Agile and Waterfall

Programming Languages: Scala, Python, Linux Shell scripting, T-SQL, SQL, DAX, Power Query (M)

Database Systems: MS SQL, MS-Access, MySQL, SSAS TABULAR, Hbase, Cassandra.

Data Warehouse Tools: Management Studio/Enterprise Manager, Query Analyzer, SSIS,SSAS, SQL Profiler.

Operating Systems: Windows 2003/2008/ XP/Vista/7/8/10 Server, UNIX, Linux(Ubuntu), and Mac OS.

Reporting Tools: Power BI, Excel, SQL Server Reporting Services 2012/2016.

Web Technologies: JDBC, XML

Tools: MS SQL Developer, AZURE, Microsoft Office suite desktop applications (e.g. MS Visual Studio, Word, Excel, PowerPoint), MS Power BI, PowerApps, MS Flow, Toad

PROFESSIONAL EXPERIENCE:

Confidential

Sr Consultant Big Data Engineer

Responsibilities:

  • Created extract jobs using Sqoop for both history and incremental data to migrate and process large data
  • Performed data cleanse and enrichment using Spark with performance optimization.
  • Experienced in Hive, created managed and external tables; performed optimization; used partitioning and bucketing; used hive vectorization; used explain plan for query plan and reviewing stages, used various join types.
  • Experienced in ingestion and analysis of structured and unstructured data using Spark and SparkSQL.
  • Experienced in real time streaming with Kafka, Flume and Spark Streaming. Used streamingcontext with both Basic sources like files /sockets and advance sources like kafka and Flume for live data integration and processing.
  • Loaded and transformed large sets of structured, semi structured and unstructured data.
  • Performance optimization of Sqoop jobs.
  • Defined job flow using Oozie.
  • Managed and reviewed Hadoop log files.
  • Worked with Columnar NoSQL (HBase and Cassandra) database.
  • Supported Map Reduce Programs those are running on the cluster.
  • Involved in loading data from UNIX file system to HDFS.
  • Installed and configured Hive and written Hive UDFs.
  • Worked closely with multiple Data Systems to access information for testing

Environment: Cloudera, Hadoop, Spark, Hive, Linux, HBase, Sqoop, scala, Oozie, SQL Server.

Confidential, Richardson, TX

Big Data/ SQL BI/Power BI Engineer

Responsibilities:

  • Designed, engineered and built data platform solutions using Big Data Technologies
  • Established and communicated fit for purpose analytical platforms for business prototypes.
  • Used Sqoop to dump data from relational database into HDFS for processing.
  • Performance optimization of Sqoop jobs.
  • Imported and exported full and incremental data into HDFS and Hive using Sqoop.
  • Explored, investigated, recommended, and implemented data-centric technologies for the platform by running POCs for streaming data with kafka, flume and sparkstreaming.
  • Used Scala and Hive in the analysis of data.
  • Experienced in managing and reviewing Hadoop log files.
  • Loaded and transformed large sets of structured, semi structured and unstructured data.
  • Managed data coming from different sources and loaded data from UNIX file system to HDFS.
  • Provided high level technical support in resolving technical issues involving Microsoft PowerBI, SQL Azure and associated products and services.
  • Executed and supported processes and jobs to meet deliverables in a timely manner per SLA.
  • Developed impressive Power BI visualizations and dashboards using Sparksql/ Impala/ hiveql connectors and managing related issues.
  • Performed Power BI desktop installation, gateway installing, and configuration.
  • Performed Power BI data modelling, adding measures and calculated columns.
  • Created and supported Power BI embedded reports.
  • Facilitated delivering of Power BI, MS Flow, SQL Server, BigData and PowerApps program curriculum for new agents SQL Connectivity
  • Provide advanced technical support to resolve issues related to SQL Server issues related to SQL Server connectivity, linked servers, data movement, and SQL BI related to SSRS.
  • Develop solutions for SQL Server issues related to Installation, Performance and troubleshooting SQL On-Premise environments.
  • Assisted customers with configuring Kerberos and Windows authentication.

Confidential, Dallas, Tx

SQL/BI Developer

Responsibilities:

  • Built complex T-SQL queries using concepts like joins, case statements and functions to retrieve and analyze data from multiple tables in the database.
  • Utilized various SQL Server Constraints (Primary Keys, Foreign Keys, Defaults, Unique keys and Check) to ensure data integrity.
  • Extensive use of T-SQL in writing Stored Procedures, Triggers, Tables, User Defined Functions, Views and Indexes.
  • Performed extraction, transformation, and loading (ETL) using SSIS
  • Created, deployed and managed C-level reports using SSRS
  • Backed up database file for each client to keep track of their business data in main database.
  • Created ad-hoc queries using T-SQL in SSMS to assist in support of users and everyday tasks.
  • Troubleshoot problems and answered questions, installing upgrades and patches
  • Assist with documentation for steps to be taken while upgrading the SQL server.
  • Assist with the implementation of SQL Logins, Roles and Authentication Modes as a part of Security Policies for various categories of users.
  • Assist with checking SQL Log, Activity log and Error Log for troubleshoot the hardware and software issue within specific time frame individually.

Confidential, Dallas, Tx

Business Analyst

Responsibilities:

  • Performed regulatory Requirement and Reporting needs.
  • Elicitation and analysis of business needs, translating them into functional requirements.
  • Interacted with various levels of authority when acting as a liaison between other departments/businesses especially senior leadership at Meridian
  • Experienced drafting requirements at all levels in a project (use-cases, functional, and non-functional requirements).
  • Facilitated requirements gathering sessions with multiple stakeholders
  • User interface prototyping for web-based products
  • Ensured reporting design and content conforms to HIPAA standards
  • Experienced working in Agile and Waterfall development methodologies
  • Performed Data Mapping for Dashboard reports
  • Performed Screen Mock ups for Customized solutions
  • Documented BRD and FRD
  • Assisted with User Acceptance Testing
  • Assisted Developers and Testers during various phases of the Project
  • Analyze QA defects and assist with Testing Team with seeking Clarification from Business

We'd love your feedback!