No matter what bullet you shoot, someone will have a story about how bad they are. It takes some experimenting to find the right powder/bullet combination. Usually, it is not the bullet. Try reducing the powder charge and see what happens. Once you find the one that works, stay with it.
When sighting a rifle in, always ALWAYS clean the barrel between each shot and let it cool some. I can't stress that enough. Run noeor two patches with bore cleaner on them and follow with one or two dry patches between each shot. Remember, in a hunting situation, 99.9% of the time, your shot is through a clean, cold barrel.
I spent a few years shooting and testing muzzleloaders for a prominent company. As a result, I have had the opportunity to shoot a lot of front-stuffers. I am not a very good shot but I do know how to shoot.
The problem you are having sounds on the surface like barrel fouling. Think about it. If the first shot is dead on, why would the second one not be and why can't you adjust the scope? Start at the muzzle and work your way down eliminating things. Then, start experimenting with loads and bullets.
Step number one is always clean the bore.
Also...it helps to be adept at cussing.