Hmm? Didn't the terrigen crystals kill people if they were touched or when the terrigen mist was released from within by breaking them? Isn't it the same terrigen mist that killed those people that got released into the oceans when the crystals deteriorated?
The diviner metal is what killed people. It literally "divined" who was genetically suited to becoming inhuman and killed anyone unworthy. When the terrigen crystals went to the bottom of the ocean, the terrigen dissolved but the diviner metal didn't. So the result is the fish oil will affect people who can become Inhumans but not affect anyone else at all.