Use bgipfs for upload #1039

azf20 · 2025-01-25T23:35:05Z

Description

To upload a site to BuidlGuidl's IPFS node, in the nextjs package:

yarn ipfs:init
yarn ipfs

These commands wrap initialisation and upload commands from bgipfs. This uses a default proxy endpoint (no auth required at this time)

technophile-04

This is great Adam!! was just testing and while running yarn ipfs I got the below error:

ipfs error

󰂃 6% ❯ yarn ipfs
  ▲ Next.js 14.2.11

   Creating an optimized production build ...

🌼   daisyUI 4.12.10
├─ ✔︎ 2 themes added             https://daisyui.com/docs/themes
╰─ ❤︎ Support daisyUI project:   https://opencollective.com/daisyui

 ✓ Compiled successfully
 ✓ Linting and checking validity of types
 ✓ Collecting page data
   Generating static pages (0/8)  [=   ](node:6527) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:6534) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:6536) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:6535) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:6537) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:6533) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
 ✓ Generating static pages (8/8)
 ✓ Collecting build traces
   Finalizing page optimization  ..(node:6609) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
 ✓ Finalizing page optimization

Route (app)                                                                  Size     First Load JS
┌ ○ /                                                                        1.65 kB         432 kB
├ ○ /_not-found                                                              879 B          89.7 kB
├ ○ /blockexplorer                                                           1.4 kB          435 kB
├ ● /blockexplorer/address/[address]                                         235 B           434 kB
├   └ /blockexplorer/address/0x0000000000000000000000000000000000000000
├ ● /blockexplorer/transaction/[txHash]                                      1.42 kB         432 kB
├   └ /blockexplorer/transaction/0x0000000000000000000000000000000000000000
└ ○ /debug                                                                   7.35 kB         438 kB
+ First Load JS shared by all                                                88.8 kB
  ├ chunks/7023-38a60bcaec288bbe.js                                          31.9 kB
  ├ chunks/fd9d1056-805ccda7fca25131.js                                      53.6 kB
  └ other shared chunks (total)                                              3.32 kB


○  (Static)  prerendered as static content
●  (SSG)     prerendered as static HTML (uses getStaticProps)

IPFS add error: {
  error: HTTPError: Request Entity Too Large

  FUNCTION_PAYLOAD_TOO_LARGE

  bom1::vhn88-1737877490716-8ee1864a251b

      at Object.errorHandler [as handleError] (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/kubo-rpc-client/dist/src/lib/core.js:81:17)
      at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async Client.fetch (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/kubo-rpc-client/dist/src/lib/http.js:80:21)
      at async KuboRPCClient.addAll (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/kubo-rpc-client/dist/src/add-all.js:21:25)
      at async Object.directory (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/ipfs-uploader/dist/NodeUploader.js:90:42)
      at async UploadCommand.run (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/bgipfs/dist/commands/upload/index.js:50:19)
      at async UploadCommand._run (/Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/@oclif/core/lib/command.js:312:22)
      at async Config.runCommand (/Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/@oclif/core/lib/config/config.js:435:25)
      at async run (/Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/@oclif/core/lib/main.js:96:16)
      at async file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/bgipfs/bin/run.js:5:1 {
    response: Response {
      status: 413,
      statusText: 'Request Entity Too Large',
      headers: Headers {
        'cache-control': 'public, max-age=0, must-revalidate',
        'content-length': '93',
        'content-type': 'text/plain; charset=utf-8',
        date: 'Sun, 26 Jan 2025 07:44:52 GMT',
        server: 'Vercel',
        'strict-transport-security': 'max-age=63072000',
        'x-matched-path': '/api/ipfs-proxy/add',
        'x-vercel-error': 'FUNCTION_PAYLOAD_TOO_LARGE',
        'x-vercel-id': 'bom1::vhn88-1737877490716-8ee1864a251b'
      },
      body: ReadableStream { locked: true, state: 'closed', supportsBYOB: true },
      bodyUsed: true,
      ok: false,
      redirected: false,
      type: 'basic',
      url: 'https://www.bgipfs.com/api/ipfs-proxy/add?stream-channels=true&wrap-with-directory=true&cid-version=1&progress=false'
    }
  },
  message: 'Request Entity Too Large\n' +
    '\n' +
    'FUNCTION_PAYLOAD_TOO_LARGE\n' +
    '\n' +
    'bom1::vhn88-1737877490716-8ee1864a251b\n',
  stack: 'HTTPError: Request Entity Too Large\n' +
    '\n' +
    'FUNCTION_PAYLOAD_TOO_LARGE\n' +
    '\n' +
    'bom1::vhn88-1737877490716-8ee1864a251b\n' +
    '\n' +
    '    at Object.errorHandler [as handleError] (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/kubo-rpc-client/dist/src/lib/core.js:81:17)\n' +
    '    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n' +
    '    at async Client.fetch (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/kubo-rpc-client/dist/src/lib/http.js:80:21)\n' +
    '    at async KuboRPCClient.addAll (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/kubo-rpc-client/dist/src/add-all.js:21:25)\n' +
    '    at async Object.directory (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/ipfs-uploader/dist/NodeUploader.js:90:42)\n' +
    '    at async UploadCommand.run (file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/bgipfs/dist/commands/upload/index.js:50:19)\n' +
    '    at async UploadCommand._run (/Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/@oclif/core/lib/command.js:312:22)\n' +
    '    at async Config.runCommand (/Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/@oclif/core/lib/config/config.js:435:25)\n' +
    '    at async run (/Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/@oclif/core/lib/main.js:96:16)\n' +
    '    at async file:///Users/shivbhonde/Desktop/github/scaffold-eth-2/packages/nextjs/node_modules/bgipfs/bin/run.js:5:1'
}
{
  success: false,
  cid: '',
  error: 'Request Entity Too Large\n' +
    '\n' +
    'FUNCTION_PAYLOAD_TOO_LARGE\n' +
    '\n' +
    'bom1::vhn88-1737877490716-8ee1864a251b\n'
}
 ›   Error: Upload failed: Upload failed

azf20 · 2025-01-26T12:48:01Z

thanks @technophile-04! that's hitting a vercel function size limit, pushed a new approach which is using a cf worker

portdeveloper · 2025-01-27T07:47:45Z

Everything works great! Amazing work @azf20 !!!

This is a lot faster than my code and it is cleaner as well!

Here is the link to my app deployed with Adam's code: https://bafybeiduwahy6nmpkpvcebtctyplxmdgjvlrpwkcnrzi6zqaizse64muna.ipfs.gateway.bgipfs.com/

And here it is after adding it to my ens records: https://pallas-athena.eth.limo/

And it just works!

technophile-04 · 2025-01-27T11:48:08Z

THanks for the updates!! Yes works great now!!

Can you run yarn install once and push those commits? Because when I switch to this branch and do yarn isntall it gives me this changes:

changes image

Also a question, is there a way we could make this just on command? Instead of it developer needing to run two separate commands? Like just yarn ipfs should do it all?

Maybe we just un-gitignore the ipfs-upload.config.json? and ship that file with SE-2 directly? Since I see there is no app specific config init and seems same config for all app? (I am sure I am missing something)

Small improvement nitpicks:

While logging the link from bigipfs CLI can we log the whole link in the end?

So that the developer doesn't need to construct the whole link himself manually?

Another nitpick maybe we could handle it in future not at all necessary now:

Unfurl are not resolving:

App link: https://bafybeihijk22a4rnuxrv26ulacy3r3qzkjcbnt25atg7aqnlwqmyddblk4.ipfs.gateway.bgipfs.com/

azf20 · 2025-01-28T14:41:16Z

OK great! Been making some changes to the underlying infrastructure so let's coordinate when this rolls out!

Pabl0cks

Great job @azf20! This is really awesome 🔥🙏

I hate to be the "Windows guy" here... tried to run yarn install and is throwing some errors :

Error on Git Bash when trying to `yarn install`

➤ YN0001: │ Error: ENOENT: no such file or directory, open
    at Object.open (node:fs:565:11)
    at Object.writeFile (node:fs:2309:6)
    at node:internal/util:442:7
    at new Promise (<anonymous>)
    at Object.writeFile (node:internal/util:428:12)
    at xze (E:\GitHub\scaffold-eth-2\.yarn\releases\yarn-3.2.3.cjs:236:193779)
    at async Promise.all (index 0)
    at async UL (E:\GitHub\scaffold-eth-2\.yarn\releases\yarn-3.2.3.cjs:236:1927
49)
    at async Gze (E:\GitHub\scaffold-eth-2\.yarn\releases\yarn-3.2.3.cjs:710:128
75)
    at async Oze (E:\GitHub\scaffold-eth-2\.yarn\releases\yarn-3.2.3.cjs:710:118
46)
➤ YN0000: └ Completed in 30s 834ms
➤ YN0000: Failed with errors in 38s 447ms

If I try to install bgipfs package manually, I get these error logs:

2025-02-02T18_17_04_844Z-debug-0.log

When I use WSL on my Windows machine, yarn install + yarn ipfs are working out of the box 👌:

✓ Configuration file initialized successfully.
✓ File uploaded. CID: bafybeidxejqgov2pso47erm4uzjxa2ddo2jao7ge2ews22mvwdi5m2jepi
🚀 Upload complete! Your site is now available at: https://gateway.bgipfs.com/ipfs/${cid}

Just as nitpicks:

cid is not getting replaced (maybe is a WSL issue)
The gateway url doesn't seem to work when I try to open it on chrome/brave

azf20 · 2025-02-03T13:49:51Z

hey thanks so much @Pabl0cks! Good catch, the gateway url hadn't been updated, and it was just printing {cid} which I agree is less friendly than it might be - pushed a fix for both of those things. I'm not sure what is required for better Windows support

portdeveloper · 2025-02-04T05:48:15Z

I believe that we should focus only on WSL for windows support as everyone is moving away from git bash to WSL.

carletex · 2025-02-04T10:09:16Z

I believe that we should focus only on WSL for windows support as everyone is moving away from git bash to WSL.

I'm not against this! Seems like a pretty common practice and will save us from a lot of trouble. We'd just to state it clearly in the Docs.

Let's see what others think too

technophile-04 · 2025-02-06T11:52:16Z

packages/nextjs/scripts/deploy-ipfs.mjs

We are not using this flie right? Lets remove it

technophile-04

Works like a charm! Thanks @azf20! Just requested a small changes at https://github.com/scaffold-eth/scaffold-eth-2/pull/1039/files#r1944592425 but other than that everything looks great!

I'm not against this! Seems like a pretty common practice and will save us from a lot of trouble. We'd just to state it clearly in the Docs.

Yup I think mentioning in docs should be sufficient 🙌

rin-st

Works great for me too!

I'm not against this! Seems like a pretty common practice and will save us from a lot of trouble. We'd just to state it clearly in the Docs.

Agree!

portdeveloper and others added 11 commits January 9, 2025 10:45

Add config for yarn ipfs

3da1015

Show errors in yarn ipfs

5fef988

Fix formatting in next.config.js

470f9c1

Update deploy-ipfs to use nifty ink

1547922

Change deploy-ipfs.js to deploy-ipfs.mjs

6026051

Remove unused stuff from yarn.lock

f7cbc7c

Fix build flag not getting detected

0c139a0

Merge branch 'main' into feat/ipfs-manual

8f06d03

Merge branch 'main' into feat/ipfs-manual

faa4150

feat: bgipfs for upload

247edb4

using proxy instead of auth

b8a3a53

technophile-04 reviewed Jan 26, 2025

View reviewed changes

replace with community endpoint (uses cf worker)

9f56153

azf20 added 2 commits January 31, 2025 16:22

update + one command

3efe720

use latest endpoint

0a8de9e

Pabl0cks reviewed Feb 2, 2025

View reviewed changes

fix url & pass through cid

b215ff9

technophile-04 changed the base branch from feat/ipfs-manual to main February 6, 2025 11:43

technophile-04 added 2 commits February 6, 2025 17:17

yarn install changes

ea4d9a6

resolve merge conflict

aafd627

technophile-04 reviewed Feb 6, 2025

View reviewed changes

packages/nextjs/scripts/deploy-ipfs.mjs

Copy link

Collaborator

technophile-04 Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not using this flie right? Lets remove it

technophile-04 approved these changes Feb 6, 2025

View reviewed changes

rin-st approved these changes Feb 6, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use bgipfs for upload #1039

Use bgipfs for upload #1039

azf20 commented Jan 25, 2025

technophile-04 left a comment

azf20 commented Jan 26, 2025

portdeveloper commented Jan 27, 2025

technophile-04 commented Jan 27, 2025

azf20 commented Jan 28, 2025

Pabl0cks left a comment •

edited

Loading

azf20 commented Feb 3, 2025

portdeveloper commented Feb 4, 2025

carletex commented Feb 4, 2025

technophile-04 Feb 6, 2025

technophile-04 left a comment

rin-st left a comment •

edited

Loading

Use bgipfs for upload #1039

Are you sure you want to change the base?

Use bgipfs for upload #1039

Conversation

azf20 commented Jan 25, 2025

Description

technophile-04 left a comment

Choose a reason for hiding this comment

azf20 commented Jan 26, 2025

portdeveloper commented Jan 27, 2025

technophile-04 commented Jan 27, 2025

azf20 commented Jan 28, 2025

Pabl0cks left a comment • edited Loading

Choose a reason for hiding this comment

azf20 commented Feb 3, 2025

portdeveloper commented Feb 4, 2025

carletex commented Feb 4, 2025

technophile-04 Feb 6, 2025

Choose a reason for hiding this comment

technophile-04 left a comment

Choose a reason for hiding this comment

rin-st left a comment • edited Loading

Choose a reason for hiding this comment

Pabl0cks left a comment •

edited

Loading

rin-st left a comment •

edited

Loading