Ray J is firing back at Kim Kardashian and Kris Jenner with a lawsuit of his own after they sued him for defamation … he says they were all in on the infamous sex tape, and it wasn’t leaked against Kim’s will.

In new legal docs, obtained by TMZ, Ray J reveals what he says actually went down with the sex tape. He claims he and Kim consensually shot a sex tape in 2003 and then in 2006 they discussed releasing it, and he says Kim insisted Kris be in charge of the release and commercial exploitation of the film.

In the lawsuit, filed by attorney Howard King, Ray J claims Kim and Kris have “spent two decades peddling the false story that the sex tape … was leaked against her will.”

In the docs, Ray says on “The Kardashians” Kim, Kris and Ye falsely accused him of sexually assaulting Kim while she was asleep, releasing revenge porn and extortion. Ray says he initiated legal proceedings and Kim, Kris and their production company entered a settlement with him where Kim agreed to pay him $6 million.

Ray says the agreement also featured a requirement barring further mention or public reference to the sex tape on “The Kardashians,” but he says almost immediately after the agreement was executed Kim, Kris, Ye and Kendall Jenner all made statements in violation of the agreement.

In the docs, Ray says he is entitled to $1 million in damages under the agreement.

Kim and Kris sued Ray J for defamation last month and here he says their lawsuit is not really about defamation … he says “it’s about publicity, power, and punishment.” Ray says they filed a baseless lawsuit to “weaponize the judicial system, shirk their contractual obligations, and sacrifice on the altar of fame.”