This is surely a wasp nest. I have a nice glass bowl full of nests like this, 2-3 cm long, I personally collected years ago from sand dunes in Lanzarote, Canary Islands. I cannot post a picture because I left my camera far from home. However, I've also seen this kind of nests, both fossils or recent, a little bit bigger, say about 10 cm.
Giorgio