如何有效地保持多个 Git 子模块同步
How to keep multiple Git submodules efficiently in sync
我正在开发一个多层应用程序，我在其中为我正在构建的每个服务分配了一个项目文件夹。设置示意如下：
.
├── ProjectA
│ ├── .git
│ ├── _framework
│ ├── backend
│ ├── config
│ ├── frontend
│ └── info
├── ProjectB
│ ├── .git
│ ├── _framework
│ ├── backend
│ ├── config
│ ├── frontend
│ └── info
└── ProjectC
├── .git
├── _framework
├── backend
├── config
├── frontend
└── info
在每个项目文件夹中，我在 _framework 文件夹中配置了一个子模块。
当我在每个文件夹中积极开发时，我也在 _framework 文件夹中进行频繁更改。我发现在所有项目中保持 _framework 子模块同步会占用大量时间。
例如：
当我在 ProjectB 中开发并在子模块中进行更改时，我将提交更改并将其推送到远程存储库。然后，当我切换到 ProjectC 时，我首先需要拉取 _framework 子模块，提交主 GIT 存储库中的更改，然后才能再次开始工作。
我知道为什么 GIT 子模块是这样设置的，但是有什么方法可以使这个过程自动化吗？因此，当我在 ProjectB 中工作并推送子模块时 - 在其他项目中，相同的子模块会自动拉取并提交到主要的本地 GIT repro?
另一种方法是参考 _framework:
- 作为主项目中的子模块（与项目 A、B 和 C 处于同一级别）
- 作为每个子项目中的符号链接 (
-> ../framework_)
这样，只有主项目需要跟踪 framework_ 版本。
根据@jingx 的建议，我决定在 ProjectA、ProjectB、ProjectC 的 git 个存储库中制作一个预推送挂钩。预推挂钩似乎是一个合乎逻辑的事件，因为当您更改并成功测试 Git 子模块中的代码时，无论如何您都必须将子模块的新状态推送到主存储库中。
作为 Git 钩子的参考，我使用了来自 Atlassian 的出色解释：https://www.atlassian.com/git/tutorials/git-hooks
因为我对 JavaScript 比 Bash 脚本更熟悉，所以我决定创建一个 NodeJS 脚本作为预推送挂钩执行：
#!/usr/bin/env node
const fs = require('fs');
const path = require('path');
const util = require('util');
// Logging something to the console so that we can verify if the hook is triggered or not
console.log('Invoking pre-push hook v1.0');
// Global variables
const exec = util.promisify(require('child_process').exec);
const gitRepositoryPathOs = process.argv[1].replace(/(.*)\/\.git\/.*/, '');
const gitRepositoryParentFolder = path.dirname(gitRepositoryPathOs);
const gitRepositoryFolderName = path.basename(gitRepositoryPathOs);
const scriptFilePathOs = __filename;
const scriptLogFolderPathOs = `${path.dirname(scriptFilePathOs)}/logs`;
const pushSiblingProjects = false;
const debugRoutine = true;
let debugInfo = "";
let stdIn = [];
// Defines all the project folder names where this hook should be working with
const projectFolderNames = ['ProjectA', 'ProjectB', 'ProjectC'];
// Defines the submodules that this routine should be checking for in each project folder
const submodulePaths = ['_framework'];
/**
* Executes a shell command
*
* @param {string} cmd Shell command to execute
* @param {string} [workingDirectory=gitRepositoryParentFolder] Directory to execute the shell command in
* @returns The result of the shell command
*/
const executeShellCommand = async (cmd, workingDirectory = gitRepositoryPathOs) => {
// if (debugRoutine) debugInfo += `- executeShellCommand('${cmd}', '${workingDirectory}')\n`;
const {stdout, stderr} = await exec(cmd, {
cwd: workingDirectory
});
return stdout;
}
/**
* Starts the logic of this GIT hook routine
*
* @returns Exit code to indicate to the GIT process if it should continue or not
*/
const initGitHookRoutine = async () => {
// Global variables
let shellCommand = '';
let shellResult = '';
let siblingGitRepositoryUpdateExecuted = false;
// Catch all parameters passed to the process.argv
debugInfo += "Passed arguments:\n\n";
process.argv.forEach(function (val, index, array) {
debugInfo += `${index}: ${val}\n`;
});
debugInfo += `Standard In:\n${stdIn.join()}`;
debugInfo += "\n\n";
debugInfo += `- gitRepositoryPathOs: '${gitRepositoryPathOs}'\n`;
debugInfo += `- gitRepositoryParentFolder: '${gitRepositoryParentFolder}'\n`;
debugInfo += `- gitRepositoryFolderName: '${gitRepositoryFolderName}'\n`;
debugInfo += `- scriptFilePathOs: '${scriptFilePathOs}'\n`;
try {
// Retrieve a list of objects that we are about to push to the remote repository
shellCommand = `git diff --stat --cached origin/master`;
shellResult = await executeShellCommand(shellCommand);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
// Mark which submodules we need to process
let submodulePathsToProcess = [];
submodulePaths.forEach((submodulePath) => {
if (shellResult.indexOf(submodulePath) > -1) {
submodulePathsToProcess.push(submodulePath);
}
})
debugInfo += `- submodulePathsToProcess: ${submodulePathsToProcess.join()}\n`;
if (submodulePathsToProcess.length > 0) {
let submodulePath = '';
// Now loop through the other projects and update the submodules there
// Using "old fashioned loop style" here as it seems to work better with async function calls...
for (let i = 0; i < projectFolderNames.length; i++) {
const projectFolderName = projectFolderNames[i];
if (projectFolderName !== gitRepositoryFolderName) {
const siblingGitRepositoryPathOs = `${gitRepositoryParentFolder}/${projectFolderName}`;
debugInfo += `- processing GIT repository '${siblingGitRepositoryPathOs}'\n`;
// Loop through the submodules that we need to update
for (let j = 0; j < submodulePathsToProcess.length; j++) {
submodulePath = submodulePathsToProcess[j];
const siblingGitRepositorySubmodulePathOs = `${siblingGitRepositoryPathOs}/${submodulePath}`;
debugInfo += `- processing GIT submodule '${siblingGitRepositorySubmodulePathOs}'\n`;
try {
// Pull the latest version of the submodule from the remote repository
shellCommand = `git pull origin master`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositorySubmodulePathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
}
// Use git status to check which submodules need to be committed
try {
shellCommand = `git status`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositoryPathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
// Now check for each submodule if it needs to be committed to the sibling project repository
for (let j = 0; j < submodulePathsToProcess.length; j++) {
submodulePath = submodulePathsToProcess[j];
if (shellResult.indexOf(`${submodulePath} (new commits)`) > -1) {
// 1) Add the submodule reference to the local git staging area
try {
shellCommand = `git add ${submodulePath}`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositoryPathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
// 2) Commit the submodule reference to the local git repository
if (shellResult.indexOf('ERROR') === -1) {
siblingGitRepositoryUpdateExecuted = true;
try {
shellCommand = `git commit -m "Submodule ${path.basename(submodulePath)} updated by GIT hook utility"`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositoryPathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
}
// 3) Optionally push this to the remote repository (not recommended)
if (pushSiblingProjects) {
if (shellResult.indexOf('ERROR') === -1) {
try {
shellCommand = `git push origin master`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositoryPathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
}
}
}
}
}
}
// Check if we need to execute anything after we have modified the sibling repositories
if (siblingGitRepositoryUpdateExecuted) {
// Put your logic in here
}
}
// Dump the debug information to the console
if (debugRoutine) console.log(`* debugInfo: ${debugInfo}`);
// Dump the debug information in a log file so that we can inspect it later on
try {
fs.writeFileSync(`${scriptLogFolderPathOs}/pre-push.log`, debugInfo);
} catch (err) {
console.log(`ERROR: write the log file in folder '${scriptLogFolderPathOs}', error details: ${JSON.stringify(err)}`);
}
// To avoid push from taking place exit a code > 0
return process.exit(0);
};
/**
* This is where the execution starts
* First we capture the content from the shell standard in and then we start the logic itself
*/
// NOTE: use 'npm install split --save-dev' to install the split module
process.stdin.pipe(require('split')()).on('data', (line) => {
// Push the line into the stdIn array so we can use it later on
if (line.trim() !== '') stdIn.push(line);
}).on('end', initGitHookRoutine)
NodeJS 脚本需要一个外部模块 ('split') 来解析来自 shell 命令的标准输入，Git 挂钩将在调用脚本时添加该命令。
代码还是有点粗糙，不过我觉得这已经足够其他人扩展了。
为了让 NodeJS 脚本工作，您需要将 NodeJS 脚本的文件权限设置为包括执行（请参阅 Atlassian 页面）。
在我的 Mac 上，脚本拒绝 运行 因为 hashbang #!/usr/bin/env node 拒绝执行。我设法通过使用 sudo ln -s /usr/bin/node /usr/local/bin/node
创建指向节点可执行文件的符号链接来解决这个问题
在我实际创建符号链接之前，我首先必须在我的机器上禁用系统完整性保护。更多信息在这里：
https://www.imore.com/el-capitan-system-integrity-protection-helps-keep-malware-away
我正在开发一个多层应用程序，我在其中为我正在构建的每个服务分配了一个项目文件夹。设置示意如下：
.
├── ProjectA
│ ├── .git
│ ├── _framework
│ ├── backend
│ ├── config
│ ├── frontend
│ └── info
├── ProjectB
│ ├── .git
│ ├── _framework
│ ├── backend
│ ├── config
│ ├── frontend
│ └── info
└── ProjectC
├── .git
├── _framework
├── backend
├── config
├── frontend
└── info
在每个项目文件夹中，我在 _framework 文件夹中配置了一个子模块。
当我在每个文件夹中积极开发时，我也在 _framework 文件夹中进行频繁更改。我发现在所有项目中保持 _framework 子模块同步会占用大量时间。
例如：
当我在 ProjectB 中开发并在子模块中进行更改时，我将提交更改并将其推送到远程存储库。然后，当我切换到 ProjectC 时，我首先需要拉取 _framework 子模块，提交主 GIT 存储库中的更改，然后才能再次开始工作。
我知道为什么 GIT 子模块是这样设置的，但是有什么方法可以使这个过程自动化吗？因此，当我在 ProjectB 中工作并推送子模块时 - 在其他项目中，相同的子模块会自动拉取并提交到主要的本地 GIT repro?
另一种方法是参考 _framework:
- 作为主项目中的子模块（与项目 A、B 和 C 处于同一级别）
- 作为每个子项目中的符号链接 (
-> ../framework_)
这样，只有主项目需要跟踪 framework_ 版本。
根据@jingx 的建议，我决定在 ProjectA、ProjectB、ProjectC 的 git 个存储库中制作一个预推送挂钩。预推挂钩似乎是一个合乎逻辑的事件，因为当您更改并成功测试 Git 子模块中的代码时，无论如何您都必须将子模块的新状态推送到主存储库中。
作为 Git 钩子的参考，我使用了来自 Atlassian 的出色解释：https://www.atlassian.com/git/tutorials/git-hooks
因为我对 JavaScript 比 Bash 脚本更熟悉，所以我决定创建一个 NodeJS 脚本作为预推送挂钩执行：
#!/usr/bin/env node
const fs = require('fs');
const path = require('path');
const util = require('util');
// Logging something to the console so that we can verify if the hook is triggered or not
console.log('Invoking pre-push hook v1.0');
// Global variables
const exec = util.promisify(require('child_process').exec);
const gitRepositoryPathOs = process.argv[1].replace(/(.*)\/\.git\/.*/, '');
const gitRepositoryParentFolder = path.dirname(gitRepositoryPathOs);
const gitRepositoryFolderName = path.basename(gitRepositoryPathOs);
const scriptFilePathOs = __filename;
const scriptLogFolderPathOs = `${path.dirname(scriptFilePathOs)}/logs`;
const pushSiblingProjects = false;
const debugRoutine = true;
let debugInfo = "";
let stdIn = [];
// Defines all the project folder names where this hook should be working with
const projectFolderNames = ['ProjectA', 'ProjectB', 'ProjectC'];
// Defines the submodules that this routine should be checking for in each project folder
const submodulePaths = ['_framework'];
/**
* Executes a shell command
*
* @param {string} cmd Shell command to execute
* @param {string} [workingDirectory=gitRepositoryParentFolder] Directory to execute the shell command in
* @returns The result of the shell command
*/
const executeShellCommand = async (cmd, workingDirectory = gitRepositoryPathOs) => {
// if (debugRoutine) debugInfo += `- executeShellCommand('${cmd}', '${workingDirectory}')\n`;
const {stdout, stderr} = await exec(cmd, {
cwd: workingDirectory
});
return stdout;
}
/**
* Starts the logic of this GIT hook routine
*
* @returns Exit code to indicate to the GIT process if it should continue or not
*/
const initGitHookRoutine = async () => {
// Global variables
let shellCommand = '';
let shellResult = '';
let siblingGitRepositoryUpdateExecuted = false;
// Catch all parameters passed to the process.argv
debugInfo += "Passed arguments:\n\n";
process.argv.forEach(function (val, index, array) {
debugInfo += `${index}: ${val}\n`;
});
debugInfo += `Standard In:\n${stdIn.join()}`;
debugInfo += "\n\n";
debugInfo += `- gitRepositoryPathOs: '${gitRepositoryPathOs}'\n`;
debugInfo += `- gitRepositoryParentFolder: '${gitRepositoryParentFolder}'\n`;
debugInfo += `- gitRepositoryFolderName: '${gitRepositoryFolderName}'\n`;
debugInfo += `- scriptFilePathOs: '${scriptFilePathOs}'\n`;
try {
// Retrieve a list of objects that we are about to push to the remote repository
shellCommand = `git diff --stat --cached origin/master`;
shellResult = await executeShellCommand(shellCommand);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
// Mark which submodules we need to process
let submodulePathsToProcess = [];
submodulePaths.forEach((submodulePath) => {
if (shellResult.indexOf(submodulePath) > -1) {
submodulePathsToProcess.push(submodulePath);
}
})
debugInfo += `- submodulePathsToProcess: ${submodulePathsToProcess.join()}\n`;
if (submodulePathsToProcess.length > 0) {
let submodulePath = '';
// Now loop through the other projects and update the submodules there
// Using "old fashioned loop style" here as it seems to work better with async function calls...
for (let i = 0; i < projectFolderNames.length; i++) {
const projectFolderName = projectFolderNames[i];
if (projectFolderName !== gitRepositoryFolderName) {
const siblingGitRepositoryPathOs = `${gitRepositoryParentFolder}/${projectFolderName}`;
debugInfo += `- processing GIT repository '${siblingGitRepositoryPathOs}'\n`;
// Loop through the submodules that we need to update
for (let j = 0; j < submodulePathsToProcess.length; j++) {
submodulePath = submodulePathsToProcess[j];
const siblingGitRepositorySubmodulePathOs = `${siblingGitRepositoryPathOs}/${submodulePath}`;
debugInfo += `- processing GIT submodule '${siblingGitRepositorySubmodulePathOs}'\n`;
try {
// Pull the latest version of the submodule from the remote repository
shellCommand = `git pull origin master`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositorySubmodulePathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
}
// Use git status to check which submodules need to be committed
try {
shellCommand = `git status`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositoryPathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
// Now check for each submodule if it needs to be committed to the sibling project repository
for (let j = 0; j < submodulePathsToProcess.length; j++) {
submodulePath = submodulePathsToProcess[j];
if (shellResult.indexOf(`${submodulePath} (new commits)`) > -1) {
// 1) Add the submodule reference to the local git staging area
try {
shellCommand = `git add ${submodulePath}`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositoryPathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
// 2) Commit the submodule reference to the local git repository
if (shellResult.indexOf('ERROR') === -1) {
siblingGitRepositoryUpdateExecuted = true;
try {
shellCommand = `git commit -m "Submodule ${path.basename(submodulePath)} updated by GIT hook utility"`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositoryPathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
}
// 3) Optionally push this to the remote repository (not recommended)
if (pushSiblingProjects) {
if (shellResult.indexOf('ERROR') === -1) {
try {
shellCommand = `git push origin master`;
shellResult = await executeShellCommand(shellCommand, siblingGitRepositoryPathOs);
} catch (err) {
shellResult = `ERROR: could not execute '${shellCommand}', error details: ${JSON.stringify(err)}`;
}
debugInfo += `- shellCommand: '${shellCommand}', shellResult: '${shellResult}'\n`;
}
}
}
}
}
}
// Check if we need to execute anything after we have modified the sibling repositories
if (siblingGitRepositoryUpdateExecuted) {
// Put your logic in here
}
}
// Dump the debug information to the console
if (debugRoutine) console.log(`* debugInfo: ${debugInfo}`);
// Dump the debug information in a log file so that we can inspect it later on
try {
fs.writeFileSync(`${scriptLogFolderPathOs}/pre-push.log`, debugInfo);
} catch (err) {
console.log(`ERROR: write the log file in folder '${scriptLogFolderPathOs}', error details: ${JSON.stringify(err)}`);
}
// To avoid push from taking place exit a code > 0
return process.exit(0);
};
/**
* This is where the execution starts
* First we capture the content from the shell standard in and then we start the logic itself
*/
// NOTE: use 'npm install split --save-dev' to install the split module
process.stdin.pipe(require('split')()).on('data', (line) => {
// Push the line into the stdIn array so we can use it later on
if (line.trim() !== '') stdIn.push(line);
}).on('end', initGitHookRoutine)
NodeJS 脚本需要一个外部模块 ('split') 来解析来自 shell 命令的标准输入，Git 挂钩将在调用脚本时添加该命令。 代码还是有点粗糙，不过我觉得这已经足够其他人扩展了。
为了让 NodeJS 脚本工作，您需要将 NodeJS 脚本的文件权限设置为包括执行（请参阅 Atlassian 页面）。
在我的 Mac 上，脚本拒绝 运行 因为 hashbang #!/usr/bin/env node 拒绝执行。我设法通过使用 sudo ln -s /usr/bin/node /usr/local/bin/node
在我实际创建符号链接之前，我首先必须在我的机器上禁用系统完整性保护。更多信息在这里： https://www.imore.com/el-capitan-system-integrity-protection-helps-keep-malware-away