How times have changed for the St George Illawarra coach, who on Saturday night will take his Dragons side into an NRL preliminary final against Wests Tigers, with the winner to play in next weekend's grand final at Telstra Stadium.
Things haven't always looked so good for the 32-year-old former Dragons hooker, whose playing career was cut short by a serious neck injury at the beginning of the 2001 season.
A disappointing start to this season threatened to bring a premature end to his coaching career as well.
"To look where we were after round four and to look here now, there's not a lot of people who would have thought we'd probably be here," Brown said.
"Fortunately for everyone the players made a decision that they thought they could get here and they've done the hard work."
Critics were calling for Brown's head following the round four loss to Canberra. They said he hadn't done a long enough apprenticeship after just one year in charge of the club's Jersey Flegg side. They said he didn't have the respect of some of the senior players due to his laid-back nature and supposed carefree attitude. Others said he simply didn't have the tactical nous to coach at the highest level.
The incident where Brown slapped star player Trent Barrett's face during a sideline dressing down in a round five loss to Manly in 2003, his first season as coach, was dragged back into the headlines despite player and coach long ago mending their fences.
Brown was even subjected to a mid-season review by the club's board, something chief executive Peter Doust said was a routine procedure, though the procedure had never been made public in years past.
Throughout it all Brown remained composed.
"A coach knows when you sign up that you sign up to do a job, and if things aren't going so well, sometimes you get the punt," Brown said after the loss to the Raiders.
"But I can't think about that because then I won't make right decisions."
Gradually the wins started to come, with Brown's cause helped immeasurably by the return of representative stars Barrett, Jason Ryles, Mark Gasnier and Matt Cooper. The players, led by Luke Bailey and Barrett, also decided to take more responsibility for their performances following a heart-to-heart session the day after the loss to Canberra.
"A few people have got egg on their faces but that's just the way it goes," Dragon's prop Ryles says of his coach.
"Browny's gone from the worst coach to the best coach in 20 weeks.
"We didn't take too much notice of that criticism when it came out."
Brown says now that his commitment to the task at hand never waned, though he admits he began to question his abilities during the club's horror start.
"Whether you're going bad or not, your desire to go to work and work hard is still the same, but obviously when your winning it's much better," Brown said.
"Every normal person at some stage has some doubt. I was lucky to get reassured by Bails (Luke Bailey) and a few of the players, the supporters, the coaching staff.
"At the end of the day, they (the players) get the accolades and I get the bagging.
"The coaching staff, when we're going good they have as big an influence as when we're going bad.
"We're all in it together and everyone stuck pretty solid which is important. We could have gone one way or the other, fortunately we went up instead of down."
Brown said he sought out some advice from a few senior coaches who had gone through similar circumstances. Among them were Brisbane coach Wayne Bennett and Parramatta mentor Brian Smith, who coached Brown during his stint at the Dragons in the early 1990s.
Ironically, Smiths' Eels side could well be Brown's combatants come grand final day.
"Every coach goes through a tough time and until you actually go through it, how you're going to react to it, and what's one or two of the keys, it's all really unknown," Brown said.
"It was good to get some advice off a few people I really respected and people that were and had been where I was at.
"I was really lucky there and you don't forget those people that give you their time during that period.
"Now that it's going well you get 8 million people patting you on the back, but at the end of the day you know who was there when you were stuck, and that's important."Like