The small animal replacement problem is the concern that certain diet changes aimed at causing less harm to the world might, in fact, cause more harm - specifically, changes that result from eating smaller animals instead of larger ones. For example, when many people see the problems with factory farming, the first meat to go is often red meat, specifically cows. Sadly, if this person increases their chicken or fish consumption even moderately, this might be a bad move ethically. There are two main factors that drive this: welfare condition and meat generated per animal.
Welfare Condition
Welfare condition is pretty simple. Some factory farmed animals are treated worse than others. More specifically, there is a pretty clear consensus, both among animal activists and animal husbandry experts, that cows are generally given a much better life than chickens. You can see below a picture of cows in a feedlot (one of the worst stages of their life) vs the chickens (the default for almost all of their life). The conditions in the chicken situation are much worse (indoors, higher density, higher bird on bird aggression) basically across the board. You can imagine that if you had to choose to be a factory farmed chicken or a factory farmed cow you would definitely pick cow.
Size of the Animal
The second factor that plays a huge role is meat generated per animal or the size of the animal. Simply, cows are much, much larger than chickens. If you eat meat for a year you affect far more chickens than cows (even if you eat a perfectly equal weight in both). Chickens generate about 5 pounds of meat per animal, where cows generate around 750 pounds of meat. This results in the average person eating a lot more chickens than cows per year.
The results are that considerably more chickens are eaten by each person and consumed in the world as a whole. The below chart shows the very strong impact that chicken has relative to other land based animals. Broadly, the smaller the animal, the more eaten for the same number of calories, thus many more fish and chickens are eaten over cows or pigs.
What it Means
The ethical implication of all this data is that if you are reducing the amount of meat you eat, the best thing to do would be to give up chicken, fish, and eggs instead of the more traditional path of giving up red meat first, or becoming a pescetarian. It also means that animal activists should be careful about encouraging changes, such as reducing red meat, as these sorts of changes might result in more chicken and fish being consumed and more animals being harmed. Instead they should consider focusing on interventions such as corporate campaigns on behalf of smaller animals such as birds or fish.