Good challenge!
The growing size of a model is a big problem for any project.
However, I feel there are some issues with your hypothesis.
I'm not a professional in machine learning, so pardon me if I ask questions that aren't related to the topic.
You hypothesise that, with a pre-defined filter, we can introduce stronger inductive bias, simply speaking.
However, I'm not sure how to choose the best filter for my project.
Introducing an inductive bias at an earlier stage of the learning model is a good way to enhance accuracy, provided everyone can choose the best filter for their own project.
> However, I'm not sure how to choose the best filter for my project.
I dont think there is a single best filter. It really depends on the type of project you have. Although one way may be to inspect the output of first few layer of a traditional cnn to see if they do edge detection or texture detection or so on
Good challenge! The growing size of a model is a big problem for any project. However, I feel there are some issues with your hypothesis. I'm not a professional in machine learning, so pardon me if I ask questions that aren't related to the topic. You hypothesise that, with a pre-defined filter, we can introduce stronger inductive bias, simply speaking. However, I'm not sure how to choose the best filter for my project. Introducing an inductive bias at an earlier stage of the learning model is a good way to enhance accuracy, provided everyone can choose the best filter for their own project.
> However, I'm not sure how to choose the best filter for my project. I dont think there is a single best filter. It really depends on the type of project you have. Although one way may be to inspect the output of first few layer of a traditional cnn to see if they do edge detection or texture detection or so on