I'm not too adept at this stuff, but a couple of possibilities come to mind...
For clarity: are you creating the tracks in JJazzLab and then exporting them into QTractor (so that at this point you are only using QTractor), or are you playing them out from JJazzlab into QTractor (with the soundfont in QTractor but the drum plugin in JJazzLab)? If your drum plugin is in JJazzLab but the other sounds are in QTractor, I would guess that all your soundfont instruments have a bit more latency from making the trip to another program. When you were using the soundfont for everything, that latency would still have existed, but you wouldn't have noticed because they were in sync with each other.
If the above is the case, you might try creating a separate bus in QTractor and using the drum plugin there rather than in JJazzLab. If you only made the tracks in JJazzLab but they are all now running in QTractor:
-maybe try creating a separate bus just for the drums and put the drum plugin on that bus rather than on the track. It could be that there is a slight difference in latency between plugin-on-bus versus plugin-on-track
-if you are using ALSA for sound, you might try using JACK instead. You can play with different settings to reduce latency with JACK in a way that I don't think you can with ALSA. (Or maybe your Linux uses Pipewire. I haven't used that yet so don't know anything about its options.)
-if you change the title of your post to include the name "QTractor," it's possible RNCBC will see it and comment. He's the one who makes QTractor and can often be found lurking on Linux Musicians.
I hope my response makes sense.
Good luck!