In the book, the source claims that Charles made the comment during a conversation with Camilla over breakfast, on the day the Duke and Duchess' engagement was announced.
According to Page Six, Charles asked, "I wonder what the children will look like?"
Camilla, was apparently taken aback by the comment, replying that the child would look "absolutely gorgeous."
Allegedly lowering his voice before responding, Charles was then said to have asked: "I mean, what do you suppose their children's complexion might be?"
A spokesman for Prince Charles' office of Clarence House denied the report, saying: "This is fiction and not worth further comment."
In a recent interview on Today, Andersen (the author of the book), said that the allegations against Charles were "benign" and had been twisted by palace couriers.
"On the morning that Meghan and Harry's engagement was announced, in a very kind of benign way, Prince Charles started to muse on what their future grandchildren might look like," Andersen said, while on the show.
"I mean, here's this beautiful biracial American woman and the world's most famous redhead. I'm a grandfather, of course, we all do this, speculate on it. But it was turned into something very toxic, it was weaponized by the 'Men in Gray' who run the palace organization. Unfortunately, by the time it got to Harry, that's the way he took it," he explained.
Following the explosive Oprah interview, the Palace released a statement on behalf of Queen Elizabeth.
"The issues raised, particularly that of race, are concerning. While some recollections may vary, they are taken very seriously and will be addressed by the family privately," the statement read.
Prince William then told a reporter, "We are very much not a racist family."