Notify when an executable dependency can't be found

Or you could get to the awkward situation that it's not being compiled and
your changes aren't being tested.