You can edit the templates for side columns in Design Layout > Layout and Themes > Template Manager.
For Products, go to the Product tab and edit
- categories_sidebar.tpl
- default_sidebar.tpl
- store_sidebar.tpl
For Blog Posts, go to the Blog tab and edit
- blog_sidebar.tpl
- blog_post_sidebar.tpl