The Man They Weren't Allowed to Call 'Dad': Inside Michael and Janet Jackson's Complicated Relationship with Father Joe

Joe Jackson, who died on Wednesday at 89, was responsible for fostering the careers of music superstars The Jackson 5, Michael Jackson and Janet Jackson.

But the late manager and father of 11 did so through tough love, which often left his children struggling to connect with him and led to an estranged relationship with Michael that lasted until the “Man in the Mirror” singer’s untimely death in 2009.

Both Michael and Janet, 52, have spoken out about their relationship with Joe throughout the years. Janet opened up about her experiences with her father in 2011, during an intimate chat with Piers Morgan for his now-canceled CNN talk show.

Michael, meanwhile, got candid about the troubles he faced with Joe in both the 2003 Martin Bashir documentary, Living with Michael Jackson and again in an emotional conversation with Rabbi Shmuley Boteach — which was published in the 2009 book, The Michael Jackson Tapes: A Tragic Icon Reveals His Soul in Intimate Conversation.

Joe himself also discussed his parenting tactics while chatting with Morgan in a rare 2013 interview.

Here are some of the biggest insights the stars gave into growing up Jackson:

Joe would hit his children if they misbehaved

Stepping out of line in the Jackson household often meant you were punished physically by Joe, Michael and Janet claimed. The late star was said to have led his home with fear and would even hold a belt in his hand when the Jackson 5 rehearsed, ready to strike anyone who stepped out of line.

Asked by Bashir how often Michael was hit by Joe, Michael said, “Too much.”

“It was more than just a belt — cords, whatever was around,” Michael remembered, detailing the “strong hate” he had towards his father during the beatings. “[He’d] throw you up against the wall as hard as you could. He would lose his temper. … I was so fast, he couldn’t catch me half the time. But when he would catch me? It was bad. It was really bad.”

Michael elaborated to Boteach, saying Joe “was very physical” throughout. “He would throw you and hit you as hard as he can,” Michael said. It’s like, ‘Oh my God, why? Where is the love? Where is the fatherhood?'”

Mother Katherine, who filed for divorce twice but never went through with the separation, would often try to stop Joe from his violent outbursts to no avail.

“She was always the one in the background when he would lose his temper — hitting us and beating us,” Michael said. “I hear it now. [Adopts female voice.] ‘Joe, no, you are going to kill them. No! No, Joe, it’s too much,’ and he would be breaking furniture and it was terrible.”

Janet felt the wrath of Joe only once. “I was very young, I remember being younger than 8,” she recalled. “I can’t remember what it was that I did. I can’t remember if I truly deserved it. … My father never touched me aside from that time.”

Even towards the end of his life, Joe justified this behavior. “Parents, they are too soft on their kids,” he told Morgan. “There’s no such thing about beating a kid. You whip them and push them over something they did and they remember that in a way that they will never do it again.”

“I had to be that way because during those times, it was hard and you had a lot of gangs there in Gary, Indiana,” Joe said. “And I had to make sure they didn’t get into any kind of trouble. … My kids were brought up in a way so they respect people and they never were on drugs. Never went to jail, weren’t in no gangs… they were professional and nice.”

He concluded: “I’m glad I was tough because look what I came out with. I came out with some kids that everybody loved all over the world. And they treated everybody right.”

Joe would also threaten to end their singing careers

Hitting his kids wasn’t the only way Joe kept his kids at bay. Michael told Boteach that Joe would hurl verbal insults and threats as well.

“And one day — I hate to repeat it — but one day he said, ‘If you guys ever stop singing I will drop you like a hot potato,’ ” Michael said. “It hurt me. You would think he would think, ‘These kids have a heart and feelings.’ Wouldn’t he think that would hurt us? If I said something like that to [my kids] that would hurt. You don’t say something like that to children and never forgot it.”

Michael was so terrified of Joe that he would often get sick in his father’s presence

At one point, Joe didn’t have to lift a finger to hurt Michael.

“I don’t think he realized to this day how scared, scared [we were],” Michael admitted to Bashir. “So scared I would regurgitate. His presence — just seeing him. Sometime I’d faint and my bodyguards would have to hold me up.”

“I have fainted in his presence many times,” Michael told Boteach. “I have thrown up in his presence because when he comes in the room and this aura comes and my stomach starts hurting and I know I am in trouble.”

The fear Michael felt around his father never went away either.

‘I am scared of my father to this day,” Michael said. “I am like an angel in front of him, like scared. One day he said to me, ‘Why are you scared of me?’ I couldn’t answer him. I felt like saying, ‘Do you know what you have done?’ [voice breaks] ‘Do you know what you have done to me?’ ”

None of his children were allowed to call Joe “Dad”

The names “Dad” and “Daddy” were strictly off limits in the Jackson household. Joe, instead, wanted his kids — Maureen, 68, Jackie, 67, Tito, 64, Jermaine, 63, La Toya, 62, Marlon, 61, Randy, 56 — to call him by his first name.

“I called him Joseph,” Janet said. “One time I tried to call him dad, and he said, ‘No, I’m Joseph. You call me Joseph. I’m Joseph to you.’ He tells you one time, you don’t do it again. So I always called him Joseph.”

“He didn’t want us to call him ‘Daddy,’ and I wanted to call him ‘Daddy’ so bad,” Michael told Bashir. “He said, ‘I’m not Daddy, I’m Joseph to you.’ ”

What was Joe’s reasoning? “You had all those kids running around calling ‘Dad, dad, dad,’ — it sounds kind of funny to me,” he said, in his defense. “I didn’t really care about what they called me, just as long as they listened to what I had to tell them to make their lives successful.”

Still, Joe wasn’t as strict on Michael and Janet as he was their siblings

In Bashir’s doc, Michael admitted that he “didn’t have it as hard” as the rest of the members of the Jackson 5.

“He used me as the example, ‘Do it like Michael,’ ” the Grammy winner explained. “And if you didn’t do it the right way, he would tear you up. Really get you. … It was always, ‘Do it like Michael, do it like Michael.’ I was nervous.”

Janet also said that she didn’t experienceJoe’s wrath as often as her siblings did.

“He’s very tough… When my brother Randy and I came along, I think my parents got pretty tired,” she explained. “Having nine kids and raising these children, my other brothers and sisters would say, ‘You guys have it so easy.’ My parents were a lot more lenient with us. I thought I had it very strict.”

Janet and Michael respected Joe and what he was able to help them accomplish

Despite everything, the Jackson siblings understood the ways Joe helped them succeed.

“He has his issues, his things, and the way that he was brought up. He’s set in his ways. I think he did the best that he could,” Janet said. “I think he did a wonderful job with us, the outcome. But the way he went about it, I don’t know if I agree with that. But we turned out okay.”

“My father means well,” she continued. “I think he means well and wants nothing but the best for his kids. I just think the way he went about certain things wasn’t the best way. But it got the job done. And that’s because of maybe how he was raised, doing what he thought was best. Not knowing any better.”

As Michael told Boteach: “God bless my father because he did some wonderful things and he was brilliant. He was a genius.”

That doesn’t mean either were particularly close with Joe… especially Michael.

Though Joe eased back on his tough love in his later years, the damage to his familial relationships was done.

“We don’t speak that much… honestly, it’s not often,” Janet said of her relationship with her father, seven years before his death.

“I used to feel [sad we weren’t closer],” she said. “Not anymore. It would have been nice. I used to go over to a friend’s house, when mother would let me go, and I’d see the relationship they had with their father. How they called him ‘Dad’ and would sit on his lap. I wished our relationship was different but I know that he loves me. There’s no question about it. I know that he loves me and he’s told me before.”

Michael found it harder to move on, despite admitting to Bashir that “I totally forgive [Joe] for all of it.”

“I can’t see him as the new man,” Michael told Boteach. “He is so different now. Time and age has changed him and he sees his grandchildren and he wants to be a better father. It is almost like the ship has sailed its course and it is so hard for me to accept this other guy that is not the guy I was raised with. I just wished he had learned that earlier. Because the scar is still there, the wound. … It affects my relationship with him today.”

Michael also made sure not to follow in his father’s footsteps

Before his death from an apparent cardiac arrest at age 50, Michael himself was father to kids Michael Joseph Jackson, Jr., 21, Paris Jackson, 20, and Prince Michael Jackson II, 16. And he made sure to be the kind of father he always wished he had.

“I always said if I ever have kids I will never behave this way,” Michael told Boteach. “I won’t touch a hair on their heads. Because people always say the abused abuse and it is not true. It is not true. I am totally the opposite. The worst I do is I make them stand in the corner for a little bit and that’s it and that’s my time out for them.”

“I don’t lay a finger on my children,” Michael added, to Bashir. “I don’t want them to ever feel that way about me… ever. … I don’t allow my children to call me ‘Michael.’ I’m ‘Daddy’ — just the opposite.”

There was on Jackson sibling who did (at least publicly) appear to stay close to Joe: LaToya

In her OWN reality show, LaToya was often joined by her father. The two got closer in their later years.

“He’s very strong but he’s older now and things are different in life,” she said in a 2013 episode. “As you get older, you change a bit and I think he’s becoming more sentimental at his age. He really is.”

A 2013 episode even had Joe telling LaToya he could call her “dad.”

“You can call me dad if you like,” he told her. “I love you. I always will. You’re a good person and I wish everybody loved you like I do.”