In basic terms, crenellate | crenelate, is a verb, and crenulate is an adjective.
OED:
crenellate | crenelate, v.
Etymology: < French crénel-er + -ate suffix3. The l has been doubled partly after crenelled; partly perhaps after assumed Latin *crēnella, diminutive of crēna.
transitive. To furnish with battlements, to embattle; to furnish with embrasures or loopholes.
1851 T. H. Turner Some Acct. Domest. Archit. I. 157 (note) Laurence de Ludlow had licence to crenellate his mansion of Stoke-Say.
crenulate, adj.
Etymology: < modern Latin crēnulātus, < crēnula , diminutive of crēna (see crena n.) + -ate suffix2. In modern French crénulé.
1. Zoology and Botany. Having the edge divided into minute rounded teeth; finely notched or scalloped: said of a leaf, a shell, etc.
1872 D. Oliver Lessons Elem. Bot. (new ed.) App. 309 Lower petal..3-lobed..lateral lobes usually..crenulate.
2. Geography. Of a shoreline: having many small irregular bays formed by the action of waves on softer rock.
1919 D. W. Johnson Shore Processes vi. 278 Early in the youth of the shoreline the curves will be changed to sharply and irregularly crenulate lines by differential wave erosion... We may call a shoreline of this character a crenulate shoreline.