How much data does a scan use


Please help me understand how to determine how much data does a scan pulls in for each security for a scan and what can I do, if anything, to minimize this amount of data to make scans more efficient. I realize that the answer would depend on, at least, aggregation period. Please point me to where this question may have already been answered.

Thank you.

Marked as spam
Posted by (Questions: 1, Answers: 2)
Asked on June 9, 2021 9:51 am
Private answer

Most of your questions are answered in the following video:

To summarize, the only thing you can do to improve the time it takes for scans to process is to apply top level filters that reduce the number of ticker symbols to only those that you would actually trade. In most cases, users should use the highest possible time frame that still encompasses the specific signals you are scanning for. Lower time frames require more processing power.

In addition to this, you will want to view the following link to understand how much historical data is included at each time frame you select for your Study Filters:

Scroll down to the very bottom and read the section titled: "Data Limitations"

By carefully adjusting each of these parameters you can arrive at a scan that runs as efficiently as possible. There is really nothing you can do in the code of your Study Filter to make things run faster. But there are certain functions that should be avoided in the code for any scan. Those functions that should be avoided are:

HighestAll(), LowestAll(), TotalSum() and you should avoid using the looping statement in Thihnkorswim which is known as "fold":

That's it. Nothing more you can do.

Marked as spam
Posted by (Questions: 39, Answers: 3664)
Answered on June 9, 2021 10:07 am
I'm trying to understand internal workings of a scan like, which bar of a symbol data does a scan start evaluating. I understand that when a scan checks for latest close to be, say 5% higher than three bars ago, it looks only at the most recent bar and a bar three bars ago; that's it. It doesn't need any more data. The scanner repeats this every aggregation period. But it's not so clear to me in other situations. For example, in the built-in study, "Afterhours_percent_change". I couldn't include the code because of character limits Where does the scan start? It has to go at least as far as yesterday's close. Then, once it has determined yesterday's close, does it simply compares close of most recent bar with yesterday's close and decide if it qualifies? If the aggregation period is 1 minute, does it always start "15 astronomical days" or is it smart enough to know just start at yesterday's close if available.
( at June 11, 2021 5:48 pm)
FYI, in my previous post, I made the word count less than 1000 but I still could not post because it complained about being too long. Maybe there is a better way to post.
( at June 11, 2021 5:50 pm)
If you are encountering a character limit this means you request very likely exceeds the complexity of the solutions provided in this Q&A Forum. The issues you are grappling with and the question you have indicate that you should be pursuing one of two paths. 1. Spend several dozen hours testing and trouble shooting to discover how things work for yourself 2. Contact TD Ameritrade support and have them address your questions and concerns. We have vastly exceeded the level of complexity for the purposes of this Q&A Forum.
( at June 11, 2021 9:24 pm)
I understand about exceeding the complexity for this forum. I'd learn to troubleshoot and would appreciate any guidance or reference on how to troubleshoot. There must be some alternative to logging. In any case, thank you for your prompt response!
( at June 12, 2021 3:22 pm)