I guess the obvious question is, "What is using all the time?".
Before you know what to do to fix anything, you need to know the root cause of the performance issue. What are the predominant wait states associated with the flashback operation? If you trace it, you will know where the time is being spent and that will give you an idea of what you can do to improve the performance.