Trying to figure out how to do this. My approaches so far haven't worked and I need ideas.
So, i have 750gb of files in a JSON format that I need to get the data from into SQL...
I wrote an app to try and do it one JSON file at a time, which was actually very quick except...at some point in the process, SQL would either timeout or just close the connection for no reason even though a timeout had not elapsed. I tried multiple different methods to deal with the bulk insert always with the same result.
I know SQL 2016 is supposed to have some native JSON support but I'm working with 2012.
Some of the JSON files are upwards of 500mb.
there are 5704 files total.
So, i have 750gb of files in a JSON format that I need to get the data from into SQL...
I wrote an app to try and do it one JSON file at a time, which was actually very quick except...at some point in the process, SQL would either timeout or just close the connection for no reason even though a timeout had not elapsed. I tried multiple different methods to deal with the bulk insert always with the same result.
I know SQL 2016 is supposed to have some native JSON support but I'm working with 2012.
Some of the JSON files are upwards of 500mb.
there are 5704 files total.