The biggest food-related risk in pregnancy is listeria. It's a dangerous bacteria, to which pregnant women are especially susceptible, that can lead to miscarriage or stillbirth.
Concern about listeria contamination is the reason that unpasteurized soft cheeses are off the menu for pregnant women. When I looked into it, I found that this is probably a sensible restriction. About 20% of listeria cases in the United States over the past 15 years were caused by raw milk soft cheese (usually queso fresco, a Mexican soft cheese).
It is worth noting, however, that this is all about unpasteurized cheese. So feel free to enjoy some Brie -- just make sure the milk it is pasteurized.