Today I did the same inlay again, albeit with slightly modified parameters, because I now actually understand what is going on and how the pieces fit together exactly.

For contrast I chose a piece of purpleheart for the pocket, which is relatively hard. The inlay is just a piece of softer construction wood, which I think is pine.

I clamped it much harder, so the soft inlay wood really squeezes into every nook and cranny. I still did not use any filler.

The result is extremely satisfying,  I would dare to say that with a tiny bit more sanding this is as perfect as I am going to get it.