Disable indexing of attachments

Is there a way to disable the indexing of attachment?
We have lots of attachments blowing up the index size.
We do not need the attachments to be searchable.

Thanks for the help in advance!
