The calculation of heat transfer in fluid flow in simple flat channels is a relatively easy task for various simulations methods. However, once the channel geometry becomes more complex, numerical simulations become a bottleneck in optimizing wall geometries. We present a combination of accurate numerical simulations of arbitrary, non-flat channels and machine learning models predicting drag coefficient and Stanton number. We show that convolutional neural networks can accurately predict the target properties at a fraction of the time of numerical simulations. We use the CNN models in a virtual high-throughput screening approach to explore a large number of possible, randomly generated wall architectures. We find that S-shaped channel geometries are Pareto-optimal, a result which seems intuitive, but was not obvious before analysing the data. The general approach is not only applicable to simple flow setups as presented here, but can be extended to more complex tasks, such as multiphase or even reactive unit operations in chemical engineering.