Tech

Overview In the following article, I introduced a Gradio app using Azure virtual machines and NDLOCR. This article provides notes on how to build this app. Building the Virtual Machine To use a GPU, it was necessary to request a quota. After the request, “NC8as_T4_v3” was used for this project. Building the Docker Environment The following article was used as a reference. https://zenn.dev/koki_algebra/scraps/32ba86a3f867a4 Disabling Secure Boot The following is stated: ...

Overview This article describes how to authenticate with GakuNin RDM using Nuxt 3 and @sidebase/nuxt-auth. Demo App https://nuxt-rdm.vercel.app/ Repository https://github.com/nakamura196/nuxt-rdm Notes Initially, the following warning was displayed. AUTH_NO_ORIGIN: No origin - this is an error in production, see https://sidebase.io/nuxt-auth/resources/errors. You can ignore this during development Therefore, based on the following reference: https://auth.sidebase.io/resources/error-reference I configured it as follows, which resulted in an error. ... auth: { baseURL: process.env.NEXTAUTH_URL, }, ... The cause was that I was using an rc version of the library. ...

Overview I tried building a RAG-based chat using Azure OpenAI, LlamaIndex, and Gradio, so here are my notes. Azure OpenAI Create an Azure OpenAI resource. Then, click “Endpoint: Click here to view endpoint” to note down the endpoint and key. Then, navigate to the Azure OpenAI Service. Go to “Model catalog” and deploy “gpt-4o” and “text-embedding-3-small”. The result is displayed as follows. Downloading the Text This time, we target “The Tale of Genji” published on Aozora Bunko (a free digital library of Japanese literature). ...

Overview This is a memo about trying out the “Tale of Genji in Textbooks LOD” (Kyokasho no Naka no Genji Monogatari LOD). https://linkdata.org/work/rdf1s10294i It is described as follows. The “Tale of Genji in Textbooks LOD” is an LOD conversion of data on The Tale of Genji published in post-war authorized textbooks for the classical literature section of high schools. I would like to thank all those involved in creating and publishing the “Tale of Genji in Textbooks LOD”. ...

Overview I investigated how to use “Peripleo,” so here are my notes. “Peripleo” is described as follows. Peripleo is a browser-based tool for the mapping of things related to place. https://github.com/britishlibrary/peripleo This time, I will introduce how to use it in combination with “Rekichizu” (Historical Maps), which was introduced in the following article. Result You can try it out at the following URL. https://nakamura196.github.io/peripleo/ The repository is available here. ...

Overview I developed a simple viewer for CSV files published on the internet. You can try it at the following URL: https://nakamura196.github.io/csv_viewer/ Here is an example with a CSV file actually loaded: https://nakamura196.github.io/csv_viewer/?u=https%3A%2F%2Fraw.githubusercontent.com%2Fomeka-j%2FOmeka-S-module-BulkImport-Sample-Data%2Frefs%2Fheads%2Fmain%2Fitem.csv Repository It is published in the following repository: https://github.com/nakamura196/csv_viewer/ Summary While there are many similar services available, I hope this serves as a useful reference for quickly viewing CSV files published on the internet.

Overview I built a Gradio App using NDL Kotenseki OCR-Lite. You can try it at the following URL. https://huggingface.co/spaces/nakamura196/ndlkotenocr-lite “NDL Kotenseki OCR-Lite” provides a desktop application, so an execution environment is available without the need for a web app like Gradio. Therefore, the intended use cases for this web app include usage from smartphones or tablets, and integration via web API. Development Notes and Bug Fixes Using Submodules The original ndlkotenocr-lite was introduced as a submodule. ...

Overview I had the opportunity to try out geocoding libraries, so here are my notes. Target This time, we will use the following text as our target: 岡山市旧御野郡金山寺村。現在の岡山市金山寺。市の中心部からは直線で北方約一〇キロを隔てた金山の中腹にある。 (Okayama City, former Mino District, Kinzanji Village. Currently Kinzanji, Okayama City. Located on the hillside of Kanayama, approximately 10 kilometers north of the city center in a straight line.) Tool 1: Jageocoder - A Python Japanese geocoder First, let’s try “Jageocoder.” ...

Overview I had the opportunity to use IIIF manifest files stored in mdx.jp object storage from NestJS, so here are my notes. Background After a brief investigation into mdx.jp object storage, it appeared that CORS settings could not be configured, making it difficult to use IIIF manifest files uploaded to mdx.jp object storage directly from other viewers. https://nakamura196.pages.dev/en/posts/ad76f58db4e098/#Note (CORS permission) Therefore, we use NestJS to load the IIIF manifest files uploaded to object storage and return them. ...

Overview This is a memo on tools related to LLMs. LangChain https://www.langchain.com/ It is described as follows. LangChain is a composable framework to build with LLMs. LangGraph is the orchestration framework for controllable agentic workflows. LlamaIndex https://docs.llamaindex.ai/en/stable/ It is described as follows. LlamaIndex is a framework for building context-augmented generative AI applications with LLMs including agents and workflows. LangChain and LlamaIndex The response from gpt-4o was as follows. ...

Overview When building a chat interface using RAG (Retrieval-augmented generation) with OpenAI’s Assistants API, I used the following repository. https://github.com/openai/openai-assistants-quickstart A modification was needed regarding the handling of citation, so I am documenting it here as a memo. Background I used the above repository to try RAG with OpenAI’s Assistants API. With the default settings, citation markers like “4:13†” were displayed as-is, as shown below. Solution I modified annotateLastMessage as follows. By changing file_path to file_citation, the citation markers could be replaced. ...

Overview This article describes how to perform authentication with ORCID, OSF (The Open Science Framework), and GRDM (GakuNin RDM) using NextAuth.js. Demo Apps ORCID https://orcid-app.vercel.app/ OSF https://osf-app.vercel.app/ GRDM https://rdm-app.vercel.app/ Repository ORCID https://github.com/nakamura196/orcid_app Below is an example of the options configuration. https://github.com/nakamura196/orcid_app/blob/main/src/app/api/auth/[…nextauth]/authOptions.js export const authOptions = { providers: [ { id: "orcid", name: "ORCID", type: "oauth", clientId: process.env.ORCID_CLIENT_ID, clientSecret: process.env.ORCID_CLIENT_SECRET, authorization: { url: "https://orcid.org/oauth/authorize", params: { scope: "/authenticate", response_type: "code", redirect_uri: process.env.NEXTAUTH_URL + "/api/auth/callback/orcid", }, }, token: "https://orcid.org/oauth/token", userinfo: { url: "https://pub.orcid.org/v3.0/[ORCID]", async request({ tokens }) { const res = await fetch(`https://pub.orcid.org/v3.0/${tokens.orcid}`, { headers: { Authorization: `Bearer ${tokens.access_token}`, Accept: "application/json", }, }); return await res.json(); }, }, profile(profile) { return { id: profile["orcid-identifier"].path, // Get ORCID ID name: profile.person?.name?.["given-names"]?.value + " " + profile.person?.name?.["family-name"]?.value, email: profile.person?.emails?.email?.[0]?.email, }; }, }, ], callbacks: { async session({ session, token }) { session.accessToken = token.accessToken; session.user.id = token.orcid; // Add ORCID ID to session return session; }, async jwt({ token, account }) { if (account) { token.accessToken = account.access_token; token.orcid = account.orcid; } return token; }, }, }; OSF https://github.com/nakamura196/osf-app ...

Overview I had the opportunity to build a character detection model using YOLOv11x and the Japanese Classical Character (Kuzushiji) Dataset, so this is a memo of the process. http://codh.rois.ac.jp/char-shape/ References Previously, I performed a similar task using YOLOv5. You can check the demo and pre-trained models at the following Spaces. https://huggingface.co/spaces/nakamura196/yolov5-char Below is an example of application to publicly available images from the “National Treasure Kanazawa Bunko Documents Database.” ...

Overview We had the opportunity to train a YOLOv11 classification model (for kuzushiji/classical Japanese character recognition) using mdx.jp, so this article serves as a reference. Dataset We target the following “Kuzushiji Dataset”: http://codh.rois.ac.jp/char-shape/book/ Creating the Dataset We format the dataset to match the YOLO format. First, we merge the data, which is separated by book title, into a flat structure. #| export class Classification: def create_dataset(self, input_file_path, output_dir): # "../data/*/characters/*/*.jpg" files = glob(input_file_path) # output_dir = "../data/dataset" for file in tqdm(files): cls = file.split("/")[-2] output_file = f"{output_dir}/{cls}/{file.split('/')[-1]}" if os.path.exists(output_file): continue # print(f"Copying {file} to {output_file}") os.makedirs(f"{output_dir}/{cls}", exist_ok=True) shutil.copy(file, output_file) Next, we split the dataset using the following script: ...

Overview I had the opportunity to run a local LLM using mdx.jp’s 1GPU pack and Ollama, so this is a memo of the process. https://mdx.jp/mdx1/p/guide/charge References I referred to the following article. https://highreso.jp/edgehub/machinelearning/ollamainference.html Downloading the Model Here, we target llama3.1:70b. After the download is complete, it becomes selectable as shown below. Usage Example We use the following “Shibusawa Eiichi Biographical Materials.” https://github.com/shibusawa-dlab/lab1 Using the API Documentation was found at the following location. ...

Overview I had the opportunity to create a transparent text PDF from a PDF using Google Cloud Vision API, so this is a personal note for future reference. Below is an example of searching for simple. Background This time, we target PDFs consisting of a single page. Procedure Creating the Image Create an image to be used as the OCR target. With the default settings, the resulting image was blurry, so I set the resolution to 2x and performed position alignment considering the resolution in the process described below. ...

Overview I looked into how to use the Zotero API from Next.js, so this is a memo. As a result, I created the following application. https://zotero-rouge.vercel.app/ Library I used the following library. https://github.com/tnajdek/zotero-api-client Getting the API Key and Other Information Please refer to the following article. Usage Collection List // app/api/zotero/collections/route.js import { NextResponse } from "next/server"; import api from "zotero-api-client"; import { prisma } from "@/lib/prisma"; import { decrypt } from "../../posts/encryption"; import { getSession } from "@auth0/nextjs-auth0"; async function fetchZoteroCollections( zoteroApiKey: string, zoteroUserId: string ) { const myapi = api(zoteroApiKey).library("user", zoteroUserId); const collectionsResponse = await myapi.collections().get(); return collectionsResponse.raw; } Specific Collection // app/api/zotero/collection/[id]/route.ts import { NextResponse } from "next/server"; import api from "zotero-api-client"; import { prisma } from "@/lib/prisma"; import { decrypt } from "@/app/api/posts/encryption"; import { getSession } from "@auth0/nextjs-auth0"; async function fetchZoteroCollection( zoteroApiKey: string, zoteroUserId: string, collectionId: string ) { const myapi = api(zoteroApiKey).library("user", zoteroUserId); const collectionResponse = await myapi.collections(collectionId).get(); return collectionResponse.raw; } List of Items in a Specific Collection // app/api/zotero/collection/[id]/items/route.ts import { NextResponse, NextRequest } from "next/server"; import api from "zotero-api-client"; import { prisma } from "@/lib/prisma"; import { decrypt } from "@/app/api/posts/encryption"; import { getSession } from "@auth0/nextjs-auth0"; async function fetchZoteroCollection( zoteroApiKey: string, zoteroUserId: string, collectionId: string ) { const myapi = api(zoteroApiKey).library("user", zoteroUserId); const collectionResponse = await myapi .collections(collectionId) .items() .get(); return collectionResponse.raw; References The application is hosted on Vercel, using Vercel Postgres for the database and Prisma as the ORM. The UI was built with Tailwind CSS, using design suggestions from ChatGPT. Auth0 was adopted for authentication. ...

Overview LEAF Writer provides buttons at the top of the screen to support tag insertion. This article introduces how to customize them. As a result, I added functionality to insert <app><lem>aaa</lem><rdg>bbb</rdg></app>. https://youtu.be/XMnRP7s2atw Editing Edit the following file: packages/cwrc-leafwriter/src/components/editorToolbar/index.tsx Features for supporting tags such as person names and place names are configured as follows. For example, the description for organization has been commented out: ... const items: (MenuItem | Item)[] = [ { group: 'action', hide: isReadonly, icon: 'insertTag', onClick: () => { if (!container.current) return; const rect = container.current.getBoundingClientRect(); const posX = rect.left; const posY = rect.top + 34; showContextMenu({ // anchorEl: container.current, eventSource: 'ribbon', position: { posX, posY }, useSelection: true, }); }, title: 'Tag', tooltip: 'Add Tag', type: 'button', }, { group: 'action', type: 'divider', hide: isReadonly }, { color: entity.person.color.main, group: 'action', disabled: !isSupported('person'), hide: isReadonly, icon: entity.person.icon, onClick: () => window.writer.tagger.addEntityDialog('person'), title: 'Tag Person', type: 'iconButton', }, { color: entity.place.color.main, group: 'action', disabled: !isSupported('place'), hide: isReadonly, icon: entity.place.icon, onClick: () => window.writer.tagger.addEntityDialog('place'), title: 'Tag Place', type: 'iconButton', }, /* { color: entity.organization.color.main, group: 'action', disabled: !isSupported('organization'), hide: isReadonly, icon: entity.organization.icon, onClick: () => window.writer.tagger.addEntityDialog('organization'), title: 'Tag Organization', type: 'iconButton', }, ... As a result, the choices are limited as follows: ...

Overview GakuNin RDM provides an API at the following link. These are notes on usage examples of this API. https://api.rdm.nii.ac.jp/v2/ Reference GakuNin RDM is built on OSF (Open Science Framework), and API documentation can be found at the following link. It conforms to OpenAPI. https://developer.osf.io/ Obtaining a PAT Obtain a PAT (Personal Access Token). After logging in, you can create one from the following URL. https://rdm.nii.ac.jp/settings/tokens/ Usage You can also access it programmatically with the following script. ...

Overview This is a ChatGPT-generated answer about the differences between ShExC (ShEx Compact Syntax) and ShExJ (ShEx JSON Syntax). There may be some inaccuracies, but I hope it serves as a useful reference. Answer ShExC (ShEx Compact Syntax) and ShExJ (ShEx JSON Syntax) are both representation formats for ShEx (Shape Expressions) schemas, but they differ in notation format and use cases. The differences are explained below. 1. Notation Format ShExC (ShEx Compact Syntax): ...