linux - Comparing CSV matching rows with Python

Question

I have two CSVs containing only one column, each:

littleListIPs.csv:

10.187.172.140
10.187.172.141
10.187.172.142
10.187.172.143
10.187.172.144
10.187.172.145
10.187.172.154
10.187.172.155

(...)

-

BigListIPs.csv:

10.187.172.146
10.187.172.147
10.187.172.148
10.187.172.149
10.187.172.150
10.187.172.151
10.187.172.152
10.187.172.153
10.187.172.154
10.187.172.155

(...)

I need a script that will compare them and create a third file (output.csv), containing every single row from littleListIPs.csv, and a column that confirms if that IP exists on the BigListIPs.csv file, like in the following output (you can place ";" instead of "|"):

10.187.172.140 | Not present in BigListIPs.csv
10.187.172.141 | Not present in BigListIPs.csv
10.187.172.142 | Not present in BigListIPs.csv
10.187.172.143 | Not present in BigListIPs.csv
10.187.172.144 | Not present in BigListIPs.csv
10.187.172.145 | Not present in BigListIPs.csv
10.187.172.154 | Present in BigListIPs.csv
10.187.172.155 | Present in BigListIPs.csv

I have seen a similar case that was solved here in Stack (Python: Comparing two CSV files and searching for similar items), but I could not manipulate it well for my needs, even being a simpler case. Thanks for any help.

Answer 1

Written in python 2.x, since that's what I have handy.

• Load BigIp list into a set. Checking for in an array is O(n), checking for in a set is O(1).
• use with to open the files, which is good practice and makes sure they were closed properly.

code:

#!/usr/bin/env python

import csv

little_ip_filename = "littleListIPs.csv"
big_ip_filename = "BigListIPs.csv"
output_filename = "results.csv"

# Load all the entries from BigListIPs into a set for quick lookup.
big_ips = set()

with open(big_ip_filename, 'r') as f:
    big_ip = csv.reader(f)
    for csv_row in big_ip:
        big_ips.add(csv_row[0])

# print big_ips

with open(little_ip_filename, 'r') as input_file, open(output_filename, 'w') as output_file:
    input_csv = csv.reader(input_file)
    output_csv = csv.writer(output_file)
    for csv_row in input_csv:
        ip = csv_row[0]
        status = "Present" if ip in big_ips else "Not Present"
        output_csv.writerow([ip, status + " in BigListIPs.csv"])

littleListIPs.csv:

10.187.172.140
10.187.172.141
10.187.172.142
10.187.172.143
10.187.172.144
10.187.172.145
10.187.172.154
10.187.172.155

BigListIPs.csv:

10.187.172.146
10.187.172.147
10.187.172.148
10.187.172.149
10.187.172.150
10.187.172.151
10.187.172.152
10.187.172.153
10.187.172.154
10.187.172.155

results.csv:

10.187.172.140,Not Present in BigListIPs.csv
10.187.172.141,Not Present in BigListIPs.csv
10.187.172.142,Not Present in BigListIPs.csv
10.187.172.143,Not Present in BigListIPs.csv
10.187.172.144,Not Present in BigListIPs.csv
10.187.172.145,Not Present in BigListIPs.csv
10.187.172.154,Present in BigListIPs.csv
10.187.172.155,Present in BigListIPs.csv