I think... Go for it.
Hot tubs are supposed to be kept hot, so it'll need to be exceptionally well insulated, all round, and have a insulated cover etc.
Either that, or do a drainable one that's basically like a mini swimming pool for use just in summer that you can cover up when not in use.