There's a lot of ML research on what functions need fitting. In particular, the field of inductive logic programming – learning logical theories – should be relevant. Also, to Konrad's point, I think many people working on causal learning see connections with "mainstream" ML.