So if the nut just comes off when unscrewed (counter clockwise), it's a single action rod.
If it's a single action rod and if you tighten the nut (clockwise) as far as it'll go and you still want less relief (the neck straighter) the solution is to take the nut off and put washers under the nut. Then tighten again.
That will give you more adjustment room to straighten the neck.