Tech

Overview Memorandum for using gdrive in a LAMP environment started with Amazon Lightsail, allowing backup of files to Google Drive, etc. Procedure First, access Amazon Lightsail and press the following “Connect using SSH” button on the target instance. You can access the server as follows. Linux ip-172-26-5-202 4.19.0-19-cloud-amd64 #1 SMP Debian 4.19.232-1 (2022-03-07) x86_64 The programs included with the Debian GNU/Linux system are free software; The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the /usr/share/doc/*/copyright. The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. ___ _ _ _ _ _ | _ _ |_) |_ _ _ _ _ _ _ _ _ _ _ __ (_) | _ _ \ \ \}/ _| ' \}/ _` | ' \} |___/_|\__|_|_|\__,_|_|_|_|_|_| *** Welcome to the LAMP packaged by Bitnami 7.4.28-14 *** *** Documentation: https://docs.bitnami.com/aws/infrastructure/lamp/ *** *** https://docs.bitnami.com/aws/ *** *** Bitnami Forums: https://community.bitnami.com/ *** Last login: Thu May 12 03:25:13 2022 from 72.21.217.186 bitnami@ip-172-26-5-202:~$ Install golang Install golang as follows. ...

Overview This is a memo for using gdrive in a LAMP environment launched on Amazon Lightsail. This enables file backups to Google Drive, among other things. Steps First, access Amazon Lightsail and press the “Connect using SSH” button on the target instance. You can access the server as shown below. Linux ip-172-26-5-202 4.19.0-19-cloud-amd64 #1 SMP Debian 4.19.232-1 (2022-03-07) x86_64 The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. ___ _ _ _ | _ |_) |_ _ _ __ _ _ __ (_) | _ \ | _| ' \/ _` | ' \| | |___/_|\__|_|_|\__,_|_|_|_|_| *** Welcome to the LAMP packaged by Bitnami 7.4.28-14 *** *** Documentation: https://docs.bitnami.com/aws/infrastructure/lamp/ *** *** https://docs.bitnami.com/aws/ *** *** Bitnami Forums: https://community.bitnami.com/ *** Last login: Thu May 12 03:25:13 2022 from 72.21.217.186 bitnami@ip-172-26-5-202:~$ Installing golang Install golang as follows. ...

Overview When creating a large number of files on a shared drive, I encountered an error message “An error has occurred in Google Drive. and the file could not be saved. The cause of the above may be that the file was caught by the shared drive limitation shown below. https://support.google.com/a/answer/7338880?hl=en *The maximum number of items that can be stored on a shared drive The maximum number of items that can be stored on a shared drive is 400,000. This includes files, folders, and shortcuts. * ...

Overview When creating a large number of files in a shared drive, I encountered a situation where “An error occurred in Google Drive” was displayed and files could no longer be saved. The likely cause was hitting the following shared drive limitations. https://support.google.com/a/answer/7338880?hl=ja Maximum number of items in a shared drive A shared drive can contain a maximum of 400,000 items. This includes files, folders, and shortcuts. Daily upload limit Individual users can upload up to 750 GB per day to My Drive and all shared drives. ...

Overview gcv2hocr is a repository that converts Google Cloud Vision OCR output to hOCR format and creates searchable PDFs. https://github.com/dinosauria123/gcv2hocr I created a notebook to run the above repository on Google Colab. https://colab.research.google.com/github/nakamura196/ndl_ocr/blob/main/gcv2hocrの実行サンプル.ipynb As shown below, you can create searchable PDF files. How to Use Access the following notebook. https://colab.research.google.com/github/nakamura196/ndl_ocr/blob/main/gcv2hocrの実行サンプル.ipynb First, obtain an API key to use the Google Cloud Vision API. The following article may be helpful. https://zenn.dev/tmitsuoka0423/articles/get-gcp-api-key ...

I created a notebook that demonstrates how to delete files on Google Drive using Google Colab. I hope this is useful when you have accidentally created a large number of unnecessary files on Google Drive. https://colab.research.google.com/github/nakamura196/ndl_ocr/blob/main/Google_Drive上のファイルを削除するノートブック.ipynb

Announcements Notebook URL https://colab.research.google.com/github/nakamura196/ndl_ocr/blob/main/ndl_ocr_v2.ipynb 2022-07-06 A demo video showing how to use it has been created. https://youtu.be/46p7ZZSul0o Additionally, a ruby (furigana) text conversion feature has been added. Overview I created an NDLOCR app using Google Colab and introduced it in the following article. This time, I created Version 2, an improved version of the above notebook. You can access the notebook from the following link. https://colab.research.google.com/github/nakamura196/ndl_ocr/blob/main/ndl_ocr_v2.ipynb Features Support for multiple input formats has been added. The following options are available: ...

Overview I recently created the following article and notebook. At the time of writing the above article, only the following input format was supported. Image file mode (specified with -s f) (Use this when providing a single image file as input) However, through verification in the following article, it became clear that applying the above option to multiple images incurs significant overhead. Therefore, I modified the notebook to also support the following input format. ...

I recently wrote the following article: This time, I conducted a brief investigation on the execution time of NDLOCR using Google Colab, and here are the results. Configuration The GPU used was: Fri Apr 29 06:26:29 2022 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla V100-SXM2... Off | 00000000:00:04.0 Off | 0 | | N/A 35C P0 23W / 300W | 0MiB / 16160MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+ The following image was used. The size was 5000 x 3415 px, 1.1 MB: ...

Overview NDL Lab publishes the following automatic figure/table extraction program. https://github.com/ndl-lab/tensorflow-deeplab-v3-plus This time, I summarize how to use Google Colab for the above program, including the procedures for inputting images via Google Drive and saving results. Notebook The Google Colab notebook created this time can be accessed from the following. https://colab.research.google.com/github/nakamura196/ndl_ocr/blob/main/ndl_deeplab.ipynb By preparing a folder of input images on Google Drive, you can execute the automatic figure/table extraction process. For basic operation instructions, please check the explanations within the notebook above. Below, I introduce execution examples. ...

Overview Previously, I shared a method for running the NDLOCR app using Google Cloud Platform’s Compute Engine. However, the above method involves somewhat cumbersome procedures and incurs costs. While it is suitable for production environments, it presented a high barrier for small-scale or experimental use. To address this issue, @blue0620 created a method for running the NDLOCR app using Google Colab. https://twitter.com/blue0620/status/1519294332159012864 By using the above notebook, you can easily (with one click from “Runtime” > “Run all”) and freely run OCR. ...

Update History 2022/09/08 Updated the script descriptions to the latest version. Overview Amazon Lightsail is described as follows: Amazon Lightsail is an easy-to-use virtual private server (VPS) that makes it easy to manage cloud resources such as containers at a predictable, low price. This article introduces how to build Omeka S using Amazon Lightsail. It also covers the “custom domain” and “SSL” configuration that are generally required when making a database publicly available. ...

Overview This is a memo about running the NDLOCR application published by NDL (National Diet Library) using a virtual machine on GCP (Google Cloud Platform). For details about this application, please refer to the following repository. https://github.com/ndl-lab/ndlocr_cli Creating a VM Instance Access Compute Engine on GCP and click the “Create Instance” button at the top of the screen. Under “Machine configuration” > “Machine family”, select “GPU”. Then for “GPU type”, select “NVIDIA T4”, which is the most affordable option. Set “Number of GPUs” to 1. ...

Overview The New York Public Library provides a Digital Collections API. http://api.repo.nypl.org/ This article explains an example of how to use this API. Sign Up First, click the following link to sign up. A form like the following will be displayed, so enter the required information. After entering your information, you will receive an email with the subject Welcome to NYPL API. This email contains the Authentication Token. ...

Overview In the following article, I explained how to build an IIIF Image Server using an AWS serverless application. This time, I register a relatively large image and verify whether tile image delivery is possible. Target This time, the target is “Mining Claim Maps” (held by the University of Tokyo Komaba Library). https://iiif.dl.itc.u-tokyo.ac.jp/repo/s/ichiko/document/4120a330-2f1c-4e2c-5d48-21aed4d42704 The original image is a TIF file of nearly 300 MB. Creating Pyramidal Tiled TIFF Referencing the following site, I tried both VIPS and ImageMagick. ...

I created a repository introducing a usage example of Leaflet with Vue 3 (including coordinate range retrieval). The working example is available here: https://static.ldas.jp/vue3-leaflet/ The source code is available here: https://github.com/ldasjp8/vue3-leaflet As a Vue 3 beginner, there may be errors, but we hope this serves as a useful reference.

The following was helpful. https://stackoverflow.com/questions/59407003/set-focus-text-field-inside-dialog-when-dialog-opened By accessing $refs after a short delay when opening the dialog, it worked successfully. watch: { dialog: function(value) { if (value) { setTimeout(() => { this.$refs.name.focus(); }, 200); } } }

The explanation was found at the following link. https://develop365.gitlab.io/nuxtjs-2.8.X-doc/ja/api/configuration-watch/ export default { ..., generate: { fallback: true, }, watch: ['static'], } By providing watch in the nuxt.config.js file as shown above, the target directory became a watch target as well.

Overview As described in a previous article, I developed an application that delivers IIIF manifest files and provides the IIIF Content Search API. https://zenn.dev/nakamura196/articles/76bdc86b1b7524 However, there were parts that did not conform to the API specifications. This article shares the corrections made and introduces how to use the Presentation API Validator with practical examples. https://presentation-validator.iiif.io/ Manifest File Corrections Access the Presentation API Validator site above, enter the manifest file URL in URL of Manifest to Validate, and select the corresponding API version (in this case, 3.0). ...

I created a notebook instance using the following as a reference. https://cloud.google.com/blog/ja/topics/developers-practitioners/pytorch-google-cloud-how-train-pytorch-models-ai-platform gcloud notebooks instances create example-instance \ --vm-image-project=deeplearning-platform-release \ --vm-image-family=pytorch-1-7-cu110-notebooks \ --machine-type=n1-standard-4 \ --location=us-central1-a \ --boot-disk-size=100 \ --accelerator-core-count=1 \ --accelerator-type=NVIDIA_TESLA_T4 \ --install-gpu-driver \ --network=default At that time, I was unable to create the notebook instance due to the following error. ERROR: (gcloud.notebooks.instances.create) The zone 'projects/{project}/zones/us-central1-a' does not have enough resources available to fulfill the request. '(resource type:compute)'. Therefore, I changed the location from us-central1-a to us-central1-b, and was able to create the instance. ...