I never tried Ullman, but it is quite common that smaller wings feel more nervous. I experienced that with Duotone and Starboard V1 wings as well (Owened a 4m and still have the 3m).
At a lot spots you see that when the wind gets stronger the diff between the wind and gusts get also a lot bigger. So the smaller wings have quite often to deal with a way bigger range as bigger wings.
Besides that: a smaller wings reacts quicker on small movements as a bigger one. Once used to it, its quite fun

The big bonus: you can wing where otherwise you wouldn't be out there :)