Depends on how well your tail swivels on its pintle (Maule) or turntable (Scott/ABS).
When you depress the rudder pedal, the steering spring stretches/compresses first, then the wheel fork starts to swivel. Different winds, loads, runway camber etc have different results, so you can taxi without brakes but must think 'ahead of the curve' as spring tension must be compensated for.
The plane can certainly be taxied without brakes, but it does take some planning. Those of us that fly skis can attest to this. I've had some interesting moments taxing on skis down a slick ramp with no way to stop, trying to park in windy conditions in between airplanes and so on. Float ops present a whole other set of things to think about. It's nice in the spring to come off skis onto wheels and have brakes again, you feel sort of spoiled. If you find yourself riding the brakes all the time you may want to reevalutate your taxi technique, tire pressure, landing gear, controls set for current wind conditions etc...
In a champ and a cub, I can taxi without brakes. In my 200 hours in the maule, I need brakes to do any precise maneuvering such as lining up on a tiedown.
It's easier on the brake pads and rotor to avoid the brakes when possible, but spring tension and crosswind affect the responsiveness of rudder steering. You might want to check the tailwheel spring tension. If they're too loose, rudder steering will be less effective since less force will be applied to the tailwheel steering ears. It's good to get familiar with the absence of brakes if you fly floats or skis, but you need to use whatever controls you have available to prevent a ground loop on wheels. Since they can happen very quickly, differential brakes may be the only way to avoid one. If you're not familiar with the technique, in a strong gusty crosswind you may not catch the ground loop quickly enough to correct it.
