@albertyeah you can start anytime! First thing though is learning a programming language that can work for audio plugins.

C++ is the most common one and best supported overall while being quite tricky to get to know really well, while Rust is new and not yet so common but likely easier to start..?

So it all depends on your experience.

Anyhow, I make https://github.com/DISTRHO/DPF for dealing with plugins, but as usual documentation is a bit lacking.