I know a lot of people who are vegetarian or even vegan and have still managed to build muscle. However, you need a nutritionist or dietician to design your meal plan. If you are doing yourself make sure to eat a lot of protein rich foods, lot of healthy fats and a reasonable amount of fiber rich food.