Skyrim Special Edition also brings the power of mods to the PlayStation®4. There are less and less reason to develop for the Legendary Edition due to the inherent benefits to the enhanced game engine of SE so mods are less likely to have versions for both today. The Special Edition includes the critically acclaimed game and add-ons with all-new features like remastered art and effects, volumetric god rays, dynamic depth of field, screen-space reflections, and more. Additionally, a large limiting factor during SE's launch period was the lack of SKSE (The Skyrim script extender, used for many, if not most, mods) which has since released for SE. However this may be becoming less and less common considering owners of the Legendary Edition (Which was generally required for most large mods before the release of SE) received free upgrades to the SE when it first released. If there is an easy way to make them compatible, numerous authors choose to release and update versions for both. The article linked by Kyle Rone in his comment on your post does offer some reasons why the versions aren't compatible, but is not technically an answer to your question, since it is the other way round.

It would really depend on the mod (some could work 'out of the box' or with light modifications), but generally it is safe to assume SE mods won't work as intended for the Legendary Edition or any other version of original Skyrim.

Skyrim special edditon mods