"Anti-merge" in pandas (Python)

Question

How can I pick out the difference between to columns of the same name in two dataframes? I mean I have dataframe A with a column named X and dataframe B with column named X, if i do pd.merge(A, B, on=['X']), i'll get the common X values of A and B, but how can i get the "non-common" ones?

Answer 1

If you change the merge type to how='outer' and indicator=True this will add a column to tell you whether the values are left/both/right only:

In [2]:
A = pd.DataFrame({'x':np.arange(5)})
B = pd.DataFrame({'x':np.arange(3,8)})
print(A)
print(B)
   x
0  0
1  1
2  2
3  3
4  4
   x
0  3
1  4
2  5
3  6
4  7

In [3]:
pd.merge(A,B, how='outer', indicator=True)

Out[3]:
     x      _merge
0  0.0   left_only
1  1.0   left_only
2  2.0   left_only
3  3.0        both
4  4.0        both
5  5.0  right_only
6  6.0  right_only
7  7.0  right_only

You can then filter the resultant merged df on the _merge col:

In [4]:
merged = pd.merge(A,B, how='outer', indicator=True)
merged[merged['_merge'] == 'left_only']

Out[4]:
     x     _merge
0  0.0  left_only
1  1.0  left_only
2  2.0  left_only

You can also use isin and negate the mask to find values not in B:

In [5]:
A[~A['x'].isin(B['x'])]

Out[5]:
   x
0  0
1  1
2  2