Объединение json файлов python

Как объединить несколько файлов json в один с помощью Python

У меня есть 6 файлов json, которые я хотел бы объединить в один. Я знаю, что мне нужно использовать glob, но мне сложно понять, как это сделать. Я приложил имена файлов и код, который я пробовал. Я также создал пустой файл json с именем «merge.json», в который я хотел бы объединить все json-файлы. Все они имеют одинаковые ключи словаря, но я хотел бы просто объединить файлы, а не объединять все значения в один ключ. Я добавил, как выглядят данные и как бы они выглядели при объединении. Спасибо!

file1 = 'file1.json' . file6 = 'file6.json' 

import json import glob result = [] for f in glob.glob("*.json"): with open(f, "rb") as infile: result.append(json.load(infile)) with open("merged_file.json", "wb") as outfile: json.dump(result, outfile) 

Но я не понимаю, что входит в «*.json» и где вызываются файлы. Спасибо!

4 ответа

Возможно, вы можете попробовать, как показано ниже, проверьте код repl.it —

import glob a = glob.glob('./*.json') print (a) merged = open("merged.json", "w+") for i in a: with open(i, "r") as f: for j in f.readlines(): merged.write(j) merged.close() 

Поместите все ваши файлы JSON в один каталог и запустите этот код в том же каталоге.

import json import glob result = [] for f in glob.glob("*.json"): with open(f, "rb") as infile: result.append(json.load(infile)) with open("merged_file.json", "wb") as outfile: json.dump(result, outfile) 

Это приведет к merged_file.json который будет содержать объединенные данные из всех файлов JSON.

for f in glob.glob(«*.json») будет перебирать каждый файл json в этом каталоге в том порядке, в котором они присутствуют в каталоге.

Если вы собираетесь использовать объединенный json в качестве действительного json, вы должны хорошо его структурировать. (Предполагается, что отдельные jsons являются действительными jsons):

Работая над ответом @tdelaney:

with open("merged_file.json", "wb") as outfile: outfile.write("[") counter=1 for f in glob.glob("*.json"): with open(f, "rb") as infile: line = None for line in infile: outfile.write(line) if line is not None and not line.endswith(b"\n") outfile.write(b"\n") if counter < len(glob.glob("*.json")): outfile.write(",") else: outfile.write("]") counter=counter+1 

Давайте превратим это в полноценную рабочую программу с помощью argparse, чтобы файлы можно было указывать в командной строке. Затем решение о том, в каком каталоге хранятся желаемые файлы JSON, может быть принято во время выполнения, и вы можете использовать глобализацию оболочки, чтобы перечислить их.

#!/usr/bin/env python """Read a list of JSON files holding a list of dictionaries and merge into a single JSON file holding a list of all of the dictionaries""" import sys import argparse import json def do_merge(infiles, outfile): merged = [] for infile in infiles: with open(infile, 'r', encoding='utf-8') as infp: data = json.load(infp) assert isinstance(data, list), "invalid input" merged.extend(data) with open(outfile, 'w', encoding="utf-8") as outfp: json.dump(merged, outfp) return 0 def main(argv): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument('outfile', help="File to hold merged JSON") parser.add_argument('infiles', nargs='+', help="List of files to merge") args = parser.parse_args(argv) retval = do_merge(args.infiles, args.outfile) print(f"Merged files into ") return retval if __name__ == "__main__": retval = main(sys.argv[1:]) exit(retval) 

С образцами файлов JSON, настроенными как

~/tmp$ ./jsonmerge.py mergedjson.json mytest/*.json Merged 2 files into mergedjson.json 

Источник

How to merge two JSON files in Python

Hello Learners, today we are going to learn how to merge two JSON files in Python. Let’s see what do you know about JSON?

JSON – JavaScript Object Notation

What is a JSON file?

JSON is a file format that is used to store JavaScript objects. Now the question comes “What is a JavaScript object?”

A JavaScript object is a collection of unordered Key-Value pairs. An example of a JSON file is given below:

Here, we have 3 different files of .json type so without wasting any time let’s jump into the code to see the implementation.

Merge two JSON files without using a third file in Python

There are other methods to do this also. You can do it by importing json library but it would be a little complex for beginners who have no idea about json objects and Python dictionary. So, here we will do it using basic File Handling in Python as it will be much easier for you!

Without wasting time see the code given below:

f2data = "" with open('C:\\Users\\lenovo\\Documents\\file2.json') as f2: f2data = '\n' + f2.read() with open('C:\\Users\\lenovo\\Documents\\file1.json','a+') as f1: f1.write(f2data)

Merging two JSON files into a third file

As you have seen the image on the top, we have three JSON files and the third file ‘file3.json‘ is empty now. Let’s see what will happen after the execution of code!

f1data = f2data = "" with open('C:\\Users\\lenovo\\Documents\\file1.json') as f1: f1data = f1.read() with open('C:\\Users\\lenovo\\Documents\\file2.json') as f2: f2data = f2.read() f1data += "\n" f1data += f2data with open ('C:\\Users\\lenovo\\Documents\\file3.json', 'a') as f3: f3.write(f1data)

• In this code, we have opened the files in ‘read’ mode (which is by default) whose content we want to add in the other file.
• In both codes, we have opened file 1 and file 3 in the append mode(‘a’) respectively. Don’t you think why we didn’t use the write mode(‘w’)? If you will use write mode, it’ll replace all the existing data in the file and if you don’t want to erase the existing data you should go for append mode.
• In Python, we don’t have to think about the number of lines in the file unlike java or other languages. When you call the read method on file object like f1, f2, f3 etc, and assign it to another variable it will assign all the data of the file to that variable.

Click here to learn more about File Handling in Python.

So, that’s all for now about how to merge two JSON files in Python, till then Keep Learning, Keep Practicing, Keep Reading!

One response to “How to merge two JSON files in Python”

dude, this example is for combining json, not merging … The expected result of merging files is the following :
fruit: [], etc …

Источник

How to Merge Multiple JSON Files with Python

In this quick article, we'll focus on a few examples of how to merge multiple JSON files into a one with Python. We will answer also on those questions:

• How to merge 2 JSON files in Python?
• How to merge all JSON files in directory?
• Merge JSON files by pattern
• Keep trace on merging multiple JSON files

If you are interested in combining CSV files into DataFrame then you can check this detailed article: How to merge multiple CSV files with Python

This image illustrates the process of merging two JSON files into single on by Python:

Merge multiple JSON files into one

To merge multiple JSON files with trace into one we can use :

import pandas as pd import glob, os, json json_dir = 'data/json_files_dir' json_pattern = os.path.join(json_dir, '*.json') file_list = glob.glob(json_pattern) dfs = [] for file in file_list: with open(file) as f: json_data = pd.json_normalize(json.loads(f.read())) json_data['site'] = file.rsplit("/", 1)[-1] dfs.append(json_data) df = pd.concat(dfs) 

If you like to learn more about this code and how to customize it. Then you can check the next steps:

Step 1: List multiple JSON files in a folder

Merging multiple files requires several Python libraries like: pandas , glob , os and json .

Next we can see how to list JSON files in a folder with Python:

import pandas as pd import glob, os, json json_dir = 'data/json_files_dir' json_pattern = os.path.join(json_dir, '*.json') file_list = glob.glob(json_pattern) 

This will result in a list with the absolute JSON files like:

Note: for JSON lines you may need to change the matching pattern to '*.jl'

Step 2: Read and merge multiple JSON file into DataFrame

Finally we are going to process all JSON files found in the previous step one by one.

We are reading the files with f.read() and loading them as JSON records by method json.loads .

Finally we are going to create a Pandas DataFrame with pd.json_normalize . All DataFrames are appended to a list.

The last step is concatenating list of DataFrames into a single one by: pd.concat(dfs)

dfs = [] for file in file_list: with open(file) as f: json_data = pd.json_normalize(json.loads(f.read())) json_data['site'] = file.rsplit("/", 1)[-1] dfs.append(json_data) df = pd.concat(dfs) 

If you like to have a trace of each record from which file is coming - then you can use a line like:

json_data['site'] = file.rsplit("/", 1)[-1] 

We are converting the absolute file path in the file name so:

Python merge json files - lines

Below you can find how to merge multiple JSON line files. What we need is to add parameter lines=True :

import pandas as pd import glob, os, json json_dir = '/home/user/data' json_pattern = os.path.join(json_dir, '*.json') file_list = glob.glob(json_pattern) dfs = [] for file in file_list: df_temp = pd.read_json(file, lines=True) df_temp['source'] = file.rsplit("/", 1)[-1] dfs.append(df_temp) df = pd.concat(dfs) 

Summary

In this article we covered how to merge multiple JSON files into one in Python.

We also covered merging of all JSON files in directory by pattern in Python. As a bonus we saw how to save information into Pandas DataFrame and keep track of each source JSON file.

By using SoftHints - Python, Linux, Pandas , you agree to our Cookie Policy.

Источник