Whitelisting is setup in the jail.conf file using a space separated list.
Just add the following line containing your ip addresses
ignoreip = 127.0.0.1 192.168.1.0/24 8.8.8.8
Depending on your Linux distro this could be in one of several locations, but typically /etc/fail2ban/
