In a bleak assessment of the impact of Brexit, they suggested grocery costs would rise, businesses could go bust and round year supplies put at risk.
Customers could be left in a two-tier system that means the better off buy more expensive, British goods while those who are poorer are left with lower-standard cheap imports.
Peers said there was a "striking" contrast between Government confidence about the potential impact of Brexit on the price and availability of food and concerns raised by the industry.
The UK's ports would be choked up with delays if EU food imports are subject to the same border checks as other imported produce, the House of Lords' EU Energy and Environment sub-committee warned.
Sunderland at Wembley: London pub landlord thanks SAFC fans for ‘bumper weekend’ after League play-off final victory over Wycombe Wanderers
Sunderland AFC fans launch fundraiser as ‘thank you’ to staff at Covent Garden pub after Wembley play-off final win
Woman caught drink-driving in Sunderland Aldi car park
Sunderland to get its first roller rink in Keel Square with live DJ - here's when it opens
19 Sunderland locations hosting Queen's Platinum Jubilee street parties
Allowing them through with few checks would raise safety concerns, it added.
Peers said even though it was unclear yet what any potential agreement could have it was "inconceivable" that Brexit would have no impact on EU food imports to the UK.
"If an agreement cannot be negotiated, Brexit is likely to result in an average tariff on food imports of 22%," the report said.
"While this would not equate to a 22% increase in food prices for consumers, there can be no doubt that prices paid at the checkout would rise.
"To counteract this the Government could cut tariffs on all food imports, EU and non-EU, but this would pose a serious risk of undermining UK food producers who could not compete on price."
The committee found that it would not be possible to increase food production in time to meet any shortfall caused by Brexit.
Reductions in EU labour could lead to an increase in recruitment or higher wages for domestic workers by paying higher wages but the costs may have to be passed on to customers or some businesses "may cease to be viable".
Peers said it was unclear whether the Government's goal is maintaining or reducing food prices, or maintaining high animal welfare and food safety standards.
The report said Brexit could mean those who can afford it will buy high-quality local produce while those that cannot may have to turn to cheaper imports that "could be produced to lower standards".
Committee chairman Lord Teverson said: "Throughout our inquiry there was a striking contrast between Government confidence and industry concerns.
"The minister may not be worried about the potential for Brexit to impact on the price and availability of food, but the representatives of the food and farming industry, importers, port authorities and consumer organisations were vocal in their concerns.
"The Government has some important choices to make."
Half the UK's food is imported, with 30% from from the EU, 11% from countries with EU trade deals and the rest from other countries.
Lord Teverson added: "We are calling on the Government to set out what checks they do intend to carry out on food imports, to allow the food industry and customs authorities time to prepare and to reassure consumers that standards will be upheld."
A government spokeswoman said: "Food prices depend on a range of factors, including commodity prices, currency exchange rates, and oil prices - this will continue to be the case when we leave the EU.
"But we also want to ensure consumers have access to a wide range of food, which is why we are considering how we best manage border checks and controls when we leave the EU without impacting the smooth flow of trade.
"We will respond to the committee's report in due course."