Hard Links Not Updating when I Rerun STATA script.

Question

I have created a hard and symbolic between the output file for one STATA script to the input file for the next script. I would like that when I run the first script and the output file is updated the target file also updates as is expected with hardlinks. However, the target file never updates. I initially had my folders under dropbox, and I thought that may be the problem so I moved them to my local system but that did not help. I have read that some programs may save updates to files by removing the old one and saving a new version which may break the link. I am not sure if this is the case for STATA. Does anyone know how I can get the symbolic link to work?

Here is the Command I typed to create the links

C:\Users\[File Path]\Tasks_EMOP>mklink /H "C:\Users\[Path]\Consumption_By_Season_Cleaning\Input\emop_despenses_2011_2019_cl.dta" "C:\Users\[File Path]\Appending_Raw_Datasets\Output\emop_despenses_2011_2019_cl.dta"

// save "C:[filepath]\Consumption_By_Season_Cleaning\Input\emop_despenses_2011_2019_cl.dta", replace

* Geographical info
rename Region, lower
label var region "Region"
label define REGION 8 "Kidal", modify
rename Cercle circle 
label var circle "Cirlce" // What's this?
rename Arrond borough
label var borough "Borough"
rename Commune town
label var town "Town"
rename Milieu urban
recode urban (2=0)
label def MILIEU 0 "Rural", modify
label var urban "Urban or rural"
egen tot_exp = rowtotal(consom_total_s1 consom_total_s2 consom_total_s3 consom_total_s4), missing 
label var tot_exp "Total HH expenditure (CFA)"
 
*keep hhid year cluster region circle borough town urban tot_exp taille_men
* For 2014 and 2016 ciricle, borough, and town are completely missing
preserve
use "$input/emop_indivitus_2011_2019_cl.dta", clear
keep iid hhid year region circle borough town
collapse (firstnm)  region circle borough town, by(hhid year)
keep if year == 2011 | year == 2013 | year == 2014 | year == 2016
tempfile geo_vars
save `geo_vars'
restore
merge 1:1 hhid year using `geo_vars', update replace
drop _merge
* For R users
gen region_id = region
gen circle_id = circle 
gen borough_id =  borough
gen town_id = town
* Check duplicates
duplicates report hhid cluster region circle borough town year // None
rename hhid idmenage
save "$output/emop_despenses_2011_2019_cl.dta", replace
// save "C:\Users\[File Path]\Consumption_By_Season_Cleaning\Input\emop_despenses_2011_2019_cl.dta", replace

// save "C:[filepath]\Consumption_By_Season_Cleaning\Input\emop_despenses_2011_2019_cl.dta", replace

// save "C:[filepath]\Consumption_By_Season_Cleaning\Input\emop_despenses_2011_2019_cl.dta", replace

// save "C:[filepath]\Consumption_By_Season_Cleaning\Input\emop_despenses_2011_2019_cl.dta", replace

Answer 1

Hello there,

If you're experiencing issues with hard links not updating when rerunning a STATA script, there are a few possible explanations and solutions to consider:

Caching: It's possible that STATA or the operating system is caching the data and not recognizing the changes made to the files. In this case, try clearing the cache or restarting STATA before rerunning the script.

File permissions: Ensure that you have the necessary permissions to modify the files linked by the hard links. If you are running the STATA script with limited privileges, it may prevent the updates from being reflected in the linked files. Make sure you have the appropriate read and write permissions for the files and directories involved.

Timing and synchronization: If the hard links are updated by another process or script simultaneously, there may be conflicts or delays in the updates. Consider synchronizing the processes or scripts to ensure that the changes are applied in the correct order.

File system limitations: Some file systems may have limitations on how hard links are handled. Ensure that the file system you are using supports hard links and that there are no known limitations or issues with updating them.

Test with symbolic links: As an alternative, you can try using symbolic links instead of hard links and see if the updates are properly reflected. Symbolic links are a different type of file link that may behave differently in your specific scenario.

Use file copies instead: If updating the hard links continues to present difficulties, consider using file copies instead. Make a copy of the original file before running the script, and modify the copy during script execution. This way, you can ensure that each run of the script operates on a fresh copy of the file, eliminating any potential issues with hard link updates.

I used AI provided by ChatGPT to formulate part of this response. I have verified that the information is accurate before sharing it with you.

Hope this resolves your Query !!

--If the reply is helpful, please Upvote and Accept it as an answer–