It is definitely a known issue - in Toyotaspeak it's known as "TNF 54/16 - High Shift Effort".
I had the same issue and had the replacement parts fitted back in October and the gearshift has been good ever since. Refer to this thread http://www.newhilux.net/phpbb/viewtopic.php?f=123&p=404568
I was led to believe that it relates to a hydraulic lock problem in the shift mechanism.