Please help contribute to the Reddit categorization project here

    Stuck_In_the_Matrix

    + friends - friends
    23,139 link karma
    87,383 comment karma
    send message redditor for

    [–] What's the fastest speed I can get over ethernet cables? Stuck_In_the_Matrix 1 points ago in DataHoarder

    That's extremely generous of you. Let me check first to make sure my equipment can even utilize that.

    [–] Mega Millions is over $1B! However...[OC] Stuck_In_the_Matrix 1 points ago in dataisbeautiful

    If I win, I'm buying Reddit gold for every user who commented on Reddit in September.

    [–] Reddit September 2018 Comments are now available for download Stuck_In_the_Matrix 2 points ago in pushshift

    The current plan is early January. The main issues right now are:

    1) Securing funding to get 1-2 additional servers for redundancy

    2) Continuing to QA test and make sure that the new API is as backwards compatible as possible

    3) Setting up the database to handle the new authentication / rate limits.

    But January is the goal. I'm very confident that we can make that goal -- I'm working on two fronts at the moment. From a technical standpoint, making sure everything works well including the integration with BigQuery. From a fund-raising perspective, I'm also working on getting a solid pitch deck in place so I can secure the remainder of the $10,000 fund-raising drive. Thanks to some larger donations, I've raised approximately $2,200 of the $10k -- so $7.8k left!

    [–] Ideas for improving /r/datasets Stuck_In_the_Matrix 1 points ago in datasets

    Thank you! The ability to filter datasets and quickly drill-down to specific types of data is a great idea! We will definitely look at this and hopefully find a solution that is intuitive and useful.

    This is actually a feature / ability high up on my personal list of "wants."

    [–] Ideas for improving /r/datasets Stuck_In_the_Matrix 2 points ago in datasets

    R > Python :)

    Wow, and I just spoke so highly of you!

    [–] Is it possible that life exists outside of earth that doesn’t have DNA? Stuck_In_the_Matrix 9 points ago in AskScienceDiscussion

    I figured you were including RNA along with DNA in your question. What's interesting is what definitions we apply to anything we define as living. It's conceivable that someday through technology, we could create self-replicating machines that are capable of interacting with its environment and "procreating" in the sense that these machines could create copies of itself. These machines could even make subtle random changes (mutations) that would allow it to adapt to its environment and "evolve."

    However, these machines would be a product of human design and creation -- It would be extremely doubtful that such machines would develop on their own. But if you could consider these advanced machines a form of life, then they would fit the criteria of being defined as a life-form that doesn't use any type of DNA/RNA in its structure.

    [–] Is it possible that life exists outside of earth that doesn’t have DNA? Stuck_In_the_Matrix 23 points ago * (lasted edited a day ago) in AskScienceDiscussion

    Are you also excluding ribonucleic acid (RNA)? From my limited understanding of the chemistry involved, Silicon based life could theoretically be possible given the right conditions. In the presence of Oxygen such as Earth, Silicon is generally oxidized into inert rocks. There could be a set of conditions in an alien environment where life may be able to use silicon instead of carbon. Silicon can form chains, but they are less stable than hydrocarbon chains.

    When energy is released from carbon compounds, it is oxidized creating carbon dioxide as a byproduct. However, when silicon goes through that process, it creates silica as a byproduct and it is less easily removed. Silicon sits below carbon on the periodic table and has similar properties to carbon such as being able to form bonds with four other atoms at once. However, silicon is far less ideal compared to carbon for the creation of complex chains that are a crucial component to more complicated processes that lead to what we call "living organisms."

    [–] Reddit September 2018 Comments are now available for download Stuck_In_the_Matrix 2 points ago in pushshift

    Once the new beta API goes live, it will have much tighter integration with BigQuery for people who prefer using BigQuery to do data analysis on Reddit data!

    [–] What's the fastest speed I can get over ethernet cables? Stuck_In_the_Matrix 1 points ago in DataHoarder

    Wow! That's a very in-depth and informative answer! I'm going to read and re-read that to make sure I understand everything you wrote. Thanks so much for taking the time out to write that up!

    [–] What's the fastest speed I can get over ethernet cables? Stuck_In_the_Matrix 1 points ago in DataHoarder

    That's a good question. Relatively speaking, the way Elasticsearch works, it will query shards across nodes. The nodes will then coordinate with the master node and create the final query. So if I ask for the top 20 subreddits related to comment volume, all shards across multiple nodes will each contribute there work and then send those packets of data back to the coordinating node. The IOPS on each individual node is primarily what matters (where the NVMe drives / lots of RAM really shine). Then the shards may send 100 megs of data back to the coordinating node to construct the final results. Technically, it's just the speed of sending lump sum of data back from the shards. So hypothetically, if each shard returns back 100 megs, the faster the network connection, the quicker those large packets of data will get relayed back.

    In this hypothetical, if each node needs to send 100 megs of data back, then a connection speed of 1000 megs per second will take 1/10th of a second. If however I have a speed of 2000 megs a second, that time can get cut in half.

    [–] What's the fastest speed I can get over ethernet cables? Stuck_In_the_Matrix 1 points ago in DataHoarder

    That's an idea. I could probably do something like that. My main objective was connecting 4-8 ES nodes together so that they could exchange a lot of data between one another as quickly as possible.

    [–] Homemade rocket with parachute Stuck_In_the_Matrix 1 points ago * (lasted edited a day ago) in interestingasfuck

    You can usually pick up dry ice at most larger supermarkets. There are also a lot of dry ice vendors in and around cities. Generally it runs around $10 bucks for a ten pound block of dry ice. You should use gloves when handling dry ice, but you can juggle a small piece of dry ice in your bare hands so long as you keep it bouncing around and don't let it keep prolonged contact with your skin. Dry ice won't stick to your skin, but if you hold it in one area in your hand, it will quickly freeze your skin and cause minor frostbite. If you have a small piece of dry ice, you can bounce it around your hand without it hurting too much.

    Also, don't ever swallow it!

    You can also put dry ice in everclear alcohol and make super cold liquid -- but that will burn you quickly if you put your hand in it.

    Also, if you take some metal (like a spoon) and push it against the dry ice, it will start screaming!

    Another fun thing you can do is fill a bathtub with hot water and throw a chunk of dry ice in it. It will cause a tremendous amount of fog. Be careful not to let it accumulate in a small room as the CO2 will displace oxygen and you can suffocate. This is especially dangerous with small animals (like cats and dogs) because the CO2 will stay low to the ground and will cause small animals to suffocate over time in an enclosed area. So long as you keep the door open and don't let your cats / dogs in the room, you should be fine.

    [–] Reddit September 2018 Comments are now available for download Stuck_In_the_Matrix 5 points ago * (lasted edited a day ago) in pushshift

    This data has been sent to /u/fhoffa for upload to Google's Big Query (/r/bigquery) and should be available soon!

    Google's BigQuery tool is an excellent tool for analyzing big data and can be used to extract info for specific subreddits.

    [–] Reddit September comments have been ingested and are currently being processed. I should have a dump later tonight. Stuck_In_the_Matrix 1 points ago in pushshift

    The monthly dumps are supposed to be monthly, but there have been a lot of recent API changes on Reddit's end that I've been working with to improve the information that I dump.

    For example, Reddit now includes the author id (called author_fullname) which allows me the ability to look up the creation timestamp for that author. I now include that in my monthly dumps (author_created_utc field will give the epoch time of when the account was created).

    Eventually, my goal is to have daily dumps that lag approximately 7 days behind and a monthly file that is generated 7-8 days after the start of a new month.

    I hope that makes sense! Feel free to ask more specific questions if you have any!

    [–] What's the fastest speed I can get over ethernet cables? Stuck_In_the_Matrix 1 points ago in DataHoarder

    Sorry for the confusion. I have a 10Gbps switch that has 2 ports capable of that speed (transferring a little over 1,000 megabytes per second). What I'm looking for is the ability to transfer at a speed over 2,500 megabytes per second.

    [–] What's the fastest speed I can get over ethernet cables? Stuck_In_the_Matrix 1 points ago * (lasted edited 2 days ago) in DataHoarder

    Thanks for the corrections! The only reason I'm hesitant about fiber is that I don't know much about the terminology and how it is used -- but I should probably research it.

    My question was mainly considering twisted pair ethernet.

    Just to clarify -- I need to support up to 8 nodes where all the nodes are within 2 meters of each other and I'd like the fastest possible switch speeds / network adapters that are as inexpensive as possible. If fiber is the way to go, then I'll learn the terminology and how it works. The goal is to connect up to 8 nodes via a switch that can support 2.5 Gbps transfer but 4.0 or 5.0 Gbps would be ideal for future growth.

    Thanks again for your time!

    [–] Homemade rocket with parachute Stuck_In_the_Matrix 2 points ago * (lasted edited 2 days ago) in interestingasfuck

    You could probably put dry ice in the bottle with a hole already poked in the bottom. I doubt the dry ice would sublimate fast enough for rocket speeds, but if you also filled the bottle partially with warm/hot water, it should work fairly well. You will need more than a pin hole sized hole (think small screwdriver sized hole).

    WARNING: DON'T USE GLASS. Wear ear and eye protection. Also, treat it like a bomb that could always go off if you screw something up. Otherwise, this would be a kick-ass experiment. If you can make a rocket with this and somehow get it to explode in the air, extra points -- but beware of neighbors calling the police because dry ice plastic bottle explosions are ungodly fucking loud. If you fill a plastic bottle with 1/4th water and put dry ice in it and cap it, you will generally have around 3-10 seconds before it explodes (depends on how much dry ice and the temperature of the water and the type of bottle). Basically as soon as you cap the bottle, you better already be swinging it away from you.

    If you get one of those large plastic 5 gallon water bottles they sell at gas stations and explode one of those underwater in a pool (tie some bricks to it or a 20 pound weight), it will cause a small earthquake around the pool and probably end up cracking the concrete in the pool. We did this once as kids and it was insane.

    I'm probably on a list now.