Vaex Python csv to csv compare

I was looking for an option to compare huge db to db, finally I found one.

What is Vaex?

Vaex is a high-performance Python library for lazy Out-of-Core DataFrames (similar to Pandas), to visualize and explore big tabular datasets. It can calculate basic statistics for more than a billion rows per second. It supports multiple visualizations allowing interactive exploration of big data.

How to use Vaex for file compare?

First of all install pip install vaex, once it is completed, you are good to go, instead of going through whole documents of Vaex, I tried a shortcut to using existing functions written in panda to compare a file, like assert, equal etc., while running the program I got an error with set_index cannot be set for dataframe, so I decided to go over in detail and I found compare function in documentation. I thought of trying but I can’t find any Information in internet. So I decided to try by myself and the good news is it worked

In just 3 lines of code I can compare huge files.

import vaex

vxsrdf = vaex.read_csv(“ownership.csv”, copy_index=False, low_memory=True, encoding=”ISO-8859-1″) # source file

vxtrgdf = vaex.read_csv(“ownership1.csv”, copy_index=False, low_memory=True, encoding=”ISO-8859-1″) # trg file, report_missing=True, report_difference=True, show=10, orderby=None, column_names=None) #calling compare function to compare src vs target


Qlik View – Test Execution Qlik View Dashboard Creation from excel

First time I am trying QlikView Developer version after working with Power Pivot/ Power Query version of microsoft excel. In this blog I am trying to pull execution status from excel sheet which is stored in shared folder. It is super simple if you have excel pivot chart and pivot table knowledgement it is a very similar progress. I am still in process of understanding full advantage over Power Pivot/ Power Query using excel. I can do future post in day to day challenges I go through while using this tool and steps to over come that.

Once you have Qlik View Installed you can follow below Steps:

  • Open QlikView application
  • Go to File > New > Select file path then next popup screen and hit on Next step
  • Got to Data Presentation chose column header from data file
  • Save file as per your requirement and hit on Next Step
  • Select Chart type
  • Populate fields and logic required in dashboard/chart
  • Selections Option to filter the dashboard after creation
  • Bingo! Dashboard Created

5 Top Challenges in doing E2E testing?

Throughout my experience I have played multiple roles, one of them of the role is E2E test manager. I have listed top 5 challenge.

1) Not having enough knowledge to understand 100+ applications connected to ecosystem

2) Documents are lost/not documented for legacy applications are still connected to ecosystem

3) Testing E2E lifecycles in Agile timelines is not feasible, so we need to tailored based testing

4) Explaining tested flow with evidences/screenshot to Business is a time consuming

5) When defect ratio goes beyond 30%, Retest/ Re-execution will goes beyond the estimated timelines which derail the program live date

Create your website with
Get started