Change the CI builds to verify the pkg-config files we create actually work. Some improvements in the `.*ignore` files.