Whenever you come to NewZealand the best way to travel around is a car. If you stay a couple of months even buy one. There is so much to see, local busses only exist in the big cities and national busses only stop at the big hubs.
Renting cars is pretty expensive but many people do anyway. If you travel south to north check out relocation cars (Most travel north to south and car companies need to get there cars back to the north island.), you can get some pretty good deals for them.
Anyway going by bus is straight forward. You can buy bus passes which make multiple journeys much cheaper, busses leave at time and you can book everything using your smartphone (so comfortable).
If you want to go somewhere more remote and don’t have a car the hitch hiking culture of NewZealand provides a really nice alternative. It’s pretty safe, cheap and you get to know a lot of nice people. My first time I didn’t even wanted a ride when the nice couple stopped next to me and offered me a lift to Ohakune.
The ferry ride from between the islands is probably one of the most beautiful ferry rides you can get. Though quite expensive you have a great view of Wellington, the bay around it and you cruise around an hour through the sounds of Picton. It gorgeous in good weather. And if you are really lucky (we were) you meet dozens if dolphins swimming and jumping around the ferry.